Skip to content

Build your personal knowledge base with Trilium Notes

fix(llm): explain a failed web search to the model and the user

Elian Dorana39b99183746
diff --git a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/AI.html b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/AI.html
index 706a2a52fc..5dd9b769d7 100644
--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/AI.html
+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/AI.html
@@ -130,6 +130,15 @@
whose services refuse requests made from a web page, so they're shown disabled
there.</li>
</ul>
+<p>To use SearXNG, enable its JSON output by adding <code>json</code> to the
+ search formats in the instance's <code>settings.yml</code>. SearXNG turns
+ JSON off by default, and without it every search fails.</p><pre><code class="language-text-x-yaml">search:
+ formats:
+ - html
+ - json</code></pre>
+<p>SearXNG doesn't use API keys, so leave the API key empty. Fill it in only
+ if your instance sits behind a proxy that requires a token; Trilium sends
+ it as a bearer token in the <code>Authorization</code> header.</p>
<p>Whether web search is on is kept per chat; the search source is shared
by all chats. A choice the current model can't use stays in the menu, disabled;
hover the <span class="tn-icon bx bx-info-circle"></span> icon next to it
diff --git a/docs/User Guide/User Guide/AI.md b/docs/User Guide/User Guide/AI.md
index f630748111..49113be853 100644
--- a/docs/User Guide/User Guide/AI.md
+++ b/docs/User Guide/User Guide/AI.md
@@ -91,6 +91,17 @@ Where the search comes from is chosen under _Web search_ in the <span class="tn-
* _Use the model's built-in search_, the default, lets the model search with its provider's own search. Anthropic, OpenAI and Google Gemini models have one, as do Claude Code, Google Antigravity and OpenAI Codex. DeepSeek, Ollama, LM Studio and custom endpoints don't, so this choice is disabled for their models.
* A search provider (Brave Search, Tavily, Exa, Serper, Perplexity or SearXNG) searches through that service instead, whichever model is answering. Search providers are added in the _Search Providers_ section of the AI settings, which the <span class="tn-icon bx bx-cog"></span> button beside the _Web search_ heading in the menu opens. The browser version can't use Brave Search, Exa or Perplexity, whose services refuse requests made from a web page, so they're shown disabled there.
+To use SearXNG, enable its JSON output by adding `json` to the search formats in the instance's `settings.yml`. SearXNG turns JSON off by default, and without it every search fails.
+
+```yaml
+search:
+ formats:
+ - html
+ - json
+```
+
+SearXNG doesn't use API keys, so leave the API key empty. Fill it in only if your instance sits behind a proxy that requires a token; Trilium sends it as a bearer token in the `Authorization` header.
+
Whether web search is on is kept per chat; the search source is shared by all chats. A choice the current model can't use stays in the menu, disabled; hover the <span class="tn-icon bx bx-info-circle"></span> icon next to it to see why. Claude Code, GitHub Copilot, Google Antigravity and OpenAI Codex run their own tools, so they can't use a search provider.
In the chat, a search run through a search provider shows that provider's logo at the start of its line instead of the <span class="tn-icon bx bx-search"></span> magnifier; hover the logo to see the provider's name.
diff --git a/packages/trilium-core/src/services/llm/web_search.spec.ts b/packages/trilium-core/src/services/llm/web_search.spec.ts
index df3b409c37..e4b99ac781 100644
--- a/packages/trilium-core/src/services/llm/web_search.spec.ts
+++ b/packages/trilium-core/src/services/llm/web_search.spec.ts
@@ -108,6 +108,31 @@ describe("searchWeb", () => {
await expect(searchWeb({ provider: "brave", apiKey: "bk" }, "q")).rejects.toThrow("HTTP 500");
await expect(searchWeb({ provider: "bing", apiKey: "" }, "q")).rejects.toThrow("Unknown search provider type: bing");
});
+
+ it("names the SearXNG setting or the proxy behind a refusal, and the limiter behind a 429", async () => {
+ const searxng = (apiKey: string) => searchWeb({ provider: "searxng", apiKey, baseURL: "http://localhost:8888" }, "q");
+ const failure = async (status: number, apiKey: string) => {
+ respond({}, status);
+ const error = await searxng(apiKey).catch((e: unknown) => e);
+ expect(error).toBeInstanceOf(Error);
+ return (error as Error).message;
+ };
+
+ const formatsOnly = await failure(403, "");
+ expect(formatsOnly).toContain("search.formats");
+ expect(formatsOnly).not.toContain("API key");
+
+ const either = await failure(403, "xk");
+ expect(either).toContain("search.formats");
+ expect(either).toContain("API key");
+
+ const proxyOnly = await failure(401, "");
+ expect(proxyOnly).toContain("proxy");
+ expect(proxyOnly).not.toContain("search.formats");
+
+ expect(await failure(429, "")).toContain("limiter");
+ expect(await failure(500, "")).toBe("The search provider answered HTTP 500.");
+ });
});
describe("searchWeb overrides and sparse answers", () => {
@@ -142,7 +167,7 @@ describe("searchWeb overrides and sparse answers", () => {
expect(await run()).toMatchObject({ searchProvider: { type: "tavily", name: "tavily" } });
fetchMock.mockRejectedValueOnce("offline");
- expect(await run()).toEqual({ error: "offline" });
+ expect(await run()).toMatchObject({ error: "offline" });
});
});
@@ -150,7 +175,7 @@ describe("createWebSearchTool", () => {
const run = (tool: ReturnType<typeof createWebSearchTool>, query: string) =>
tool.execute?.({ query }, { toolCallId: "1", messages: [], context: {} });
- it("returns the query, the provider that ran it and its sources, or the failure for the model to report", async () => {
+ it("returns the query, the provider that ran it and its sources", async () => {
const tool = createWebSearchTool({ provider: "tavily", name: "My Tavily", apiKey: "tk" });
respond({ results: [ { ...PAGE, content: "Hit" } ] });
@@ -159,9 +184,27 @@ describe("createWebSearchTool", () => {
searchProvider: { type: "tavily", name: "My Tavily" },
sources: [ { ...PAGE, snippet: "Hit" } ]
});
+ });
- respond({}, 403);
- expect(await run(tool, "trilium")).toEqual({ error: expect.stringContaining("HTTP 403") });
+ it("points the model at read_web_page after any failure, and has it report one only the user can fix", async () => {
+ const readPage = expect.stringContaining("read_web_page");
+ const userFix = expect.stringMatching(/tell the user.*read_web_page/s);
+ const failure = async (setup: Parameters<typeof createWebSearchTool>[0], status?: number) => {
+ if (status) {
+ respond({}, status);
+ }
+ return await run(createWebSearchTool(setup), "q");
+ };
+
+ expect(await failure({ provider: "brave", apiKey: "bad" }, 401)).toEqual({ error: expect.any(String), instruction: userFix });
+ expect(await failure({ provider: "searxng", apiKey: "", baseURL: "http://localhost:8888" }, 403))
+ .toEqual({ error: expect.stringContaining("search.formats"), instruction: userFix });
+ expect(await failure({ provider: "searxng", apiKey: "" }))
+ .toEqual({ error: expect.stringContaining("has no address"), instruction: userFix });
+
+ const transient = await failure({ provider: "brave", apiKey: "bk" }, 500);
+ expect(transient).toEqual({ error: expect.stringContaining("HTTP 500"), instruction: readPage });
+ expect(transient).not.toEqual(expect.objectContaining({ instruction: expect.stringContaining("tell the user") }));
});
it("cancels the search request when the turn is stopped", async () => {
diff --git a/packages/trilium-core/src/services/llm/web_search.ts b/packages/trilium-core/src/services/llm/web_search.ts
index a4a2cc44cc..29076d98b2 100644
--- a/packages/trilium-core/src/services/llm/web_search.ts
+++ b/packages/trilium-core/src/services/llm/web_search.ts
@@ -35,7 +35,8 @@ const SEARCH_TIMEOUT_MS = 30_000;
/**
* The `web_search` tool for `setup`. Its result has the `sources` the chat lists under the tool's
* line and the `searchProvider` whose mark the line shows, and a failure is returned as
- * `{ error }` so the model can tell the user about it.
+ * `{ error, instruction }`: the chat shows `error` on the tool's line, and `instruction` directs the
+ * model to `read_web_page` and, for a {@link SearchSetupError}, to tell the user about it.
*/
export function createWebSearchTool(setup: WebSearchSetup) {
return tool({
@@ -51,12 +52,24 @@ export function createWebSearchTool(setup: WebSearchSetup) {
sources: await searchWeb(setup, query, options.abortSignal)
};
} catch (e) {
- return { error: e instanceof Error ? e.message : String(e) };
+ const error = e instanceof Error ? e.message : String(e);
+ return {
+ error,
+ instruction: e instanceof SearchSetupError ? `${SETUP_ERROR_INSTRUCTION} ${READ_PAGE_INSTRUCTION}` : READ_PAGE_INSTRUCTION
+ };
}
}
});
}
+const SETUP_ERROR_INSTRUCTION = "Only the user can fix this, in the search provider's settings or the search service itself. "
+ + "Don't retry the search; tell the user what the error says and how to fix it.";
+const READ_PAGE_INSTRUCTION = "The read_web_page tool fetches pages directly, without the search provider, "
+ + "so you can still use it to read a page whose URL you know or the user gives.";
+
+/** A search failure that only the user can fix, by changing the search provider or the service behind it. */
+class SearchSetupError extends Error {}
+
/**
* Run `query` against the search provider in `setup`. `signal` cancels the request along with the
* chat turn that made it.
@@ -108,12 +121,12 @@ export async function searchWeb(setup: WebSearchSetup, query: string, signal?: A
}
case "searxng": {
if (!setup.baseURL) {
- throw new Error("The SearXNG search provider has no address.");
+ throw new SearchSetupError("The SearXNG search provider has no address.");
}
const params = new URLSearchParams({ q: query, format: "json" });
const payload = await requestJson(`${trimTrailingSlashes(setup.baseURL)}/search?${params}`, {
headers: { "Accept": "application/json", ...(setup.apiKey && { "Authorization": `Bearer ${setup.apiKey}` }) }
- }, signal) as { results?: unknown[] };
+ }, signal, (status) => describeSearxngFailure(status, !!setup.apiKey)) as { results?: unknown[] };
return toSources(payload.results, "content").slice(0, MAX_RESULTS);
}
default:
@@ -222,18 +235,52 @@ function htmlToMarkdown(html: string): string {
return title ? `# ${title}\n\n${markdown}` : markdown;
}
-async function requestJson(url: string, init: RequestInit, signal: AbortSignal | undefined): Promise<unknown> {
+/**
+ * `describeFailure` names the cause of an HTTP error status the provider's API documents, which the
+ * user must fix; when it returns `undefined`, the generic message for the status applies.
+ */
+async function requestJson(
+ url: string,
+ init: RequestInit,
+ signal: AbortSignal | undefined,
+ describeFailure?: (status: number) => string | undefined
+): Promise<unknown> {
const timeout = AbortSignal.timeout(SEARCH_TIMEOUT_MS);
const response = await llmFetch(url, { ...init, signal: signal ? AbortSignal.any([ signal, timeout ]) : timeout });
if (!response.ok) {
+ const description = describeFailure?.(response.status);
+ if (description) {
+ throw new SearchSetupError(description);
+ }
if (response.status === 401 || response.status === 403) {
- throw new Error(`The search provider refused the request (HTTP ${response.status}); check its API key in the AI settings.`);
+ throw new SearchSetupError(`The search provider refused the request (HTTP ${response.status}); check its API key in the AI settings.`);
}
throw new Error(`The search provider answered HTTP ${response.status}.`);
}
return await response.json();
}
+/**
+ * The cause of a SearXNG error status. SearXNG has no authentication: on `/search` it answers 403
+ * only for a format missing from `search.formats` and 429 from its limiter, so a 401, or a 403
+ * when an API key is set, can also come from a proxy in front of the instance.
+ */
+function describeSearxngFailure(status: number, hasApiKey: boolean): string | undefined {
+ if (status === 403 && !hasApiKey) {
+ return "The SearXNG instance doesn't allow JSON results (HTTP 403); add `json` to `search.formats` in its settings.yml.";
+ }
+ if (status === 403) {
+ return "The SearXNG instance refused the request (HTTP 403). Either `json` is missing from `search.formats` in its settings.yml, or a proxy in front of it refused the API key set in the AI settings.";
+ }
+ if (status === 401) {
+ return "A proxy in front of the SearXNG instance asked for credentials (HTTP 401); check the API key set for it in the AI settings.";
+ }
+ if (status === 429) {
+ return "The SearXNG instance's limiter blocked the request (HTTP 429); turn off `server.limiter` in its settings.yml, or add Trilium's address to the limiter's `pass_ip` list.";
+ }
+ return undefined;
+}
+
/** The results that carry a web URL, read from `urlField`, with the excerpt read from `snippetField`. */
function toSources(results: unknown[] | undefined, snippetField: string, urlField = "url"): WebSearchSource[] {
const sources: WebSearchSource[] = [];