Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
perf(llm): keep the LLM stack out of the startup path
Elian Doranb4bbc501799d
diff --git a/.claude/skills/adding-llm-mcp-tools/SKILL.md b/.claude/skills/adding-llm-mcp-tools/SKILL.mdindex ca85d3549b..e54acd8b36 100644--- a/.claude/skills/adding-llm-mcp-tools/SKILL.md+++ b/.claude/skills/adding-llm-mcp-tools/SKILL.md@@ -36,9 +36,12 @@ Rule: **any tool that writes** (`setContent`, `save`, `setAttribute`, `createNew**Node-only tools take the other door.** A tool that needs something core cannot have in the browser(the in-app documentation reader, for instance) lives in `apps/server/src/services/llm/tools/`-(`doc_notes.ts`, `help_tools.ts`) and registers itself at server startup by calling-`registerToolRegistry(theRegistry)` (`index.ts:48`), which appends to the same array. Standalone-simply runs without it. Put a tool there **only** if it genuinely cannot run under sqlite-wasm —+(`doc_notes.ts`, `help_tools.ts`) and registers at server startup via+`registerToolRegistryLoader(async () => (await import("./tools/x.js")).xTools)` from core's light+`tools/registration.ts` (see `registerServerLlmExtensions`); the first chat turn or MCP request+resolves the loader through `resolveToolRegistries()` and appends the registry to the same array.+Register the loader, never a static import — the loader form is what keeps the tool stack (zod, the+AI SDK) out of the server's startup path. Standalone simply runs without it. Put a tool there **only** if it genuinely cannot run under sqlite-wasm —core is the default home, and anything placed in the server loses the standalone and desktopconsumers.diff --git a/CLAUDE.md b/CLAUDE.mdindex 0652bd09df..5b03a075a2 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -232,7 +232,7 @@ The `_hidden` subtree holds system notes with deterministic `_`-prefixed IDs so- Operations called from Express routes already have CLS context; standalone services (schedulers, Electron IPC handlers) do not### Adding New LLM Tools-Tools are `defineTools({...})` registries in `packages/trilium-core/src/services/llm/tools/` (`note_tools`, `attribute_tools`, `attachment_tools`, `hierarchy_tools`, …), served to both the LLM chat and the MCP server. Each tool: `description`, Zod `inputSchema`, `execute`, `mutates: true` for writes. A new module goes into `allToolRegistries` in `tools/index.ts`; Node-only tools (in-app docs) live in `apps/server/src/services/llm/tools/` and use `registerToolRegistry()` from `apps/server/src/services/llm/index.ts`. Add the client label under `llm.tools.<tool_name>` in imperative tense ("Search notes"). Mirror ETAPI's field choices but **don't import ETAPI mappers** — inline them so the LLM layer stays decoupled.+Tools are `defineTools({...})` registries in `packages/trilium-core/src/services/llm/tools/` (`note_tools`, `attribute_tools`, `attachment_tools`, `hierarchy_tools`, …), served to both the LLM chat and the MCP server. Each tool: `description`, Zod `inputSchema`, `execute`, `mutates: true` for writes. A new module goes into `allToolRegistries` in `tools/index.ts`; Node-only tools (in-app docs) live in `apps/server/src/services/llm/tools/` and register via `registerToolRegistryLoader()` (a lazy dynamic-import loader — see `registerServerLlmExtensions` in `apps/server/src/services/llm/index.ts`) so the tool stack stays out of the startup path. Add the client label under `llm.tools.<tool_name>` in imperative tense ("Search notes"). Mirror ETAPI's field choices but **don't import ETAPI mappers** — inline them so the LLM layer stays decoupled.### Server-Side Static AssetsNode-side assets (templates, translations, prompts) live in `apps/server/src/assets/` and are read via `RESOURCE_DIR` from `apps/server/src/services/resource_dir.ts` (`path.join(RESOURCE_DIR, "llm", "prompts", …)`); assets core itself reads (`schema.sql`, LLM skills) live in `packages/trilium-core/src/assets/`. **Never resolve paths with `import.meta.url`/`fileURLToPath` or `__dirname` + relative path** — the server is bundled to CJS and both point at the bundle, not the source tree.diff --git a/apps/server/src/routes/api/llm_chat.ts b/apps/server/src/routes/api/llm_chat.tsindex b4937a6880..929c733b13 100644--- a/apps/server/src/routes/api/llm_chat.ts+++ b/apps/server/src/routes/api/llm_chat.ts@@ -1,5 +1,4 @@import type { LlmMessage } from "@triliumnext/commons";-import { runChat } from "@triliumnext/core/src/services/llm/chat.js";import type { LlmProviderConfig } from "@triliumnext/core/src/services/llm/types.js";import type { Request, Response } from "express";@@ -46,6 +45,10 @@ async function streamChat(req: Request, res: Response) {res.on("close", () => abortController.abort());try {+ // Imported here rather than at module scope so the chat pipeline and+ // the provider SDKs land in lazy chunks (the same convention as+ // getProviderModels in core's routes/api/llm.ts).+ const { runChat } = await import("@triliumnext/core/src/services/llm/chat.js");for await (const chunk of runChat(messages, config, abortController.signal)) {res.write(`data: ${JSON.stringify(chunk)}\n\n`);// Flush immediately to ensure real-time streamingdiff --git a/apps/server/src/routes/mcp.ts b/apps/server/src/routes/mcp.tsindex 5084626e03..1ea9f1f76e 100644--- a/apps/server/src/routes/mcp.ts+++ b/apps/server/src/routes/mcp.ts@@ -8,13 +8,11 @@* arrangement outright rather than layering on top of it.*/-import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";import { getLog, options as optionService } from "@triliumnext/core";import type express from "express";import rateLimit from "express-rate-limit";import etapiTokenService from "../services/etapi_tokens.js";-import { createMcpServer } from "../services/mcp/mcp_server.js";export function register(app: express.Application) {const mcpRateLimiter = createMcpRateLimiter();@@ -124,7 +122,15 @@ function mcpGuard(req: express.Request, res: express.Response, next: express.Nexasync function handleMcpRequest(req: express.Request, res: express.Response) {try {- const server = createMcpServer();+ // Imported here rather than at module scope so the MCP SDK and the+ // tool stack land in lazy chunks: the route registers at startup, but+ // the guard above answers unauthenticated traffic before any of this+ // loads.+ const [{ StreamableHTTPServerTransport }, { createMcpServer }] = await Promise.all([+ import("@modelcontextprotocol/sdk/server/streamableHttp.js"),+ import("../services/mcp/mcp_server.js")+ ]);+ const server = await createMcpServer();const transport = new StreamableHTTPServerTransport({sessionIdGenerator: undefined, // stateless// Off because the guard above has already required a bearer token, which thediff --git a/apps/server/src/services/llm/index.spec.ts b/apps/server/src/services/llm/index.spec.tsindex 8f5aa7ce20..e63c8ee4b5 100644--- a/apps/server/src/services/llm/index.spec.ts+++ b/apps/server/src/services/llm/index.spec.ts@@ -4,20 +4,22 @@ const registrations = vi.hoisted(() => ({hostProviders: [] as { type: string; factory: () => unknown }[],docNoteReader: undefined as ((note: unknown) => string | null) | undefined,skillReader: undefined as ((file: string) => string | null) | undefined,- toolRegistries: [] as unknown[]+ toolRegistryLoaders: [] as (() => Promise<unknown>)[]}));-vi.mock("@triliumnext/core/src/services/llm/index.js", () => ({+vi.mock("@triliumnext/core/src/services/llm/host_providers.js", () => ({registerHostProvider: (type: string, factory: () => unknown) => { registrations.hostProviders.push({ type, factory }); }}));-vi.mock("@triliumnext/core/src/services/llm/skills.js", () => ({+vi.mock("@triliumnext/core/src/services/llm/skills.js", async (importOriginal) => ({+ ...await importOriginal<typeof import("@triliumnext/core/src/services/llm/skills.js")>(),registerSkillReader: (reader: (file: string) => string | null) => { registrations.skillReader = reader; }}));-vi.mock("@triliumnext/core/src/services/llm/tools/helpers.js", () => ({+vi.mock("@triliumnext/core/src/services/llm/tools/helpers.js", async (importOriginal) => ({+ ...await importOriginal<typeof import("@triliumnext/core/src/services/llm/tools/helpers.js")>(),registerDocNoteHtmlReader: (reader: (note: unknown) => string | null) => { registrations.docNoteReader = reader; }}));-vi.mock("@triliumnext/core/src/services/llm/tools/index.js", () => ({- registerToolRegistry: (registry: unknown) => { registrations.toolRegistries.push(registry); }+vi.mock("@triliumnext/core/src/services/llm/tools/registration.js", () => ({+ registerToolRegistryLoader: (loader: () => Promise<unknown>) => { registrations.toolRegistryLoaders.push(loader); }}));import { registerServerLlmExtensions } from "./index.js";@@ -26,21 +28,24 @@ import { CopilotAgentProvider } from "./providers/copilot_agent.js";import { helpTools } from "./tools/help_tools.js";describe("registerServerLlmExtensions", () => {- it("hands core every piece of the stack that needs Node", () => {+ it("hands core every piece of the stack that needs Node", async () => {registerServerLlmExtensions();// The subscription providers, which shell out to the Claude Code and Copilot- // CLIs, under the types core knows them by. Registered as factories, so- // nothing is constructed until a chat asks for it.+ // CLIs, under the types core knows them by. Registered as async factories+ // that import their module on first use, so neither the provider nor its+ // agent SDK loads until a chat asks for it.expect(registrations.hostProviders.map(p => p.type)).toEqual(["claude-agent", "copilot-agent"]);- expect(registrations.hostProviders[0].factory()).toBeInstanceOf(ClaudeAgentProvider);- expect(registrations.hostProviders[1].factory()).toBeInstanceOf(CopilotAgentProvider);+ expect(await registrations.hostProviders[0].factory()).toBeInstanceOf(ClaudeAgentProvider);+ expect(await registrations.hostProviders[1].factory()).toBeInstanceOf(CopilotAgentProvider);// The User Guide reader, which core's note-content helper calls for doc notes.expect(registrations.docNoteReader).toBeTypeOf("function");// The skill sheets: core owns the catalog and the tool, the server only// knows how to get a sheet off disk.expect(registrations.skillReader?.("search_syntax.md")).toContain("#");- // The one registry still backed by files this build alone carries.- expect(registrations.toolRegistries).toEqual([helpTools]);+ // The one registry still backed by files this build alone carries,+ // registered as a loader so the tool stack stays out of the startup path.+ expect(registrations.toolRegistryLoaders).toHaveLength(1);+ expect(await registrations.toolRegistryLoaders[0]()).toBe(helpTools);});});diff --git a/apps/server/src/services/llm/index.ts b/apps/server/src/services/llm/index.tsindex 451c0e52e8..4f0a9d9592 100644--- a/apps/server/src/services/llm/index.ts+++ b/apps/server/src/services/llm/index.ts@@ -12,27 +12,29 @@* one more line here and one more entry in core's HOST_PROVIDED_TYPES.*/-import { registerHostProvider } from "@triliumnext/core/src/services/llm/index.js";+import { registerHostProvider } from "@triliumnext/core/src/services/llm/host_providers.js";import { registerSkillReader } from "@triliumnext/core/src/services/llm/skills.js";import { registerDocNoteHtmlReader } from "@triliumnext/core/src/services/llm/tools/helpers.js";-import { registerToolRegistry } from "@triliumnext/core/src/services/llm/tools/index.js";+import { registerToolRegistryLoader } from "@triliumnext/core/src/services/llm/tools/registration.js";import { loadSkillSheet } from "../../core_assets.js";-import { ClaudeAgentProvider } from "./providers/claude_agent.js";-import { CopilotAgentProvider } from "./providers/copilot_agent.js";import { getDocNoteHtml } from "./tools/doc_notes.js";-import { helpTools } from "./tools/help_tools.js";/*** Contribute those pieces to core. Called once from startup, beside the other* registrations there, rather than run as an import side effect: core's own LLM* routes reach the stack directly, so what a chat can use must not depend on* whether some module that happens to import this one was loaded first.+ *+ * The providers and the help tools are registered as loaders, not values: their+ * modules carry the agent SDKs and the tool stack, and importing them here+ * would put all of that in the startup path of every install. The registration+ * seams are light on purpose — core resolves each loader on first use.*/export function registerServerLlmExtensions() {- registerHostProvider("claude-agent", () => new ClaudeAgentProvider());- registerHostProvider("copilot-agent", () => new CopilotAgentProvider());+ registerHostProvider("claude-agent", async () => new (await import("./providers/claude_agent.js")).ClaudeAgentProvider());+ registerHostProvider("copilot-agent", async () => new (await import("./providers/copilot_agent.js")).CopilotAgentProvider());registerDocNoteHtmlReader(getDocNoteHtml);registerSkillReader(loadSkillSheet);- registerToolRegistry(helpTools);+ registerToolRegistryLoader(async () => (await import("./tools/help_tools.js")).helpTools);}diff --git a/apps/server/src/services/llm/providers/claude_agent.ts b/apps/server/src/services/llm/providers/claude_agent.tsindex 57136e6f01..7d1cba4c7d 100644--- a/apps/server/src/services/llm/providers/claude_agent.ts+++ b/apps/server/src/services/llm/providers/claude_agent.ts@@ -634,7 +634,7 @@ export class ClaudeAgentProvider implements LlmProvider {// open port, and no dependency on the user-facing `mcpEnabled`// toggle (which exists to expose notes to *external* clients).options.mcpServers = {- trilium: { type: "sdk", name: "trilium", instance: createMcpServer() }+ trilium: { type: "sdk", name: "trilium", instance: await createMcpServer() }};// Bare server prefix auto-allows every tool from that server.allowedTools.push("mcp__trilium");diff --git a/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts b/apps/server/src/services/llm/providers/copilot_mcp_endpoint.tsindex 66d54090c6..de0360baa8 100644--- a/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts+++ b/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts@@ -105,7 +105,7 @@ async function handleRequest(}// Stateless per-request server+transport, mirroring the public /mcp route.- const mcpServer = createMcpServer();+ const mcpServer = await createMcpServer();const transport = new StreamableHTTPServerTransport({sessionIdGenerator: undefined,enableDnsRebindingProtection: true,diff --git a/apps/server/src/services/mcp/mcp_server.spec.ts b/apps/server/src/services/mcp/mcp_server.spec.tsindex 116569c9a2..4271a8c4a7 100644--- a/apps/server/src/services/mcp/mcp_server.spec.ts+++ b/apps/server/src/services/mcp/mcp_server.spec.ts@@ -52,8 +52,10 @@ const mutateRegistry: any = [["mutate_tool", { description: "m", inputSchema: {}, mutates: true, execute: mutateExecute }]];+const resolveToolRegistriesMock = vi.fn(async () => {});vi.mock("@triliumnext/core/src/services/llm/tools/index.js", () => ({- allToolRegistries: [readRegistry, mutateRegistry]+ allToolRegistries: [readRegistry, mutateRegistry],+ resolveToolRegistries: resolveToolRegistriesMock}));const { createMcpServer } = await import("./mcp_server.js");@@ -69,16 +71,17 @@ beforeEach(() => {afterEach(() => vi.restoreAllMocks());describe("createMcpServer", () => {- it("registers every tool from all registries", () => {- const server = createMcpServer();+ it("registers every tool from all registries, after resolving the lazy ones", async () => {+ const server = await createMcpServer();expect(server).toBeInstanceOf(FakeMcpServer);expect((server as any).info).toMatchObject({ name: "trilium-notes", version: "9.9.9-test" });expect(registered.map((r) => r.name)).toEqual(["read_tool", "mutate_tool"]);+ expect(resolveToolRegistriesMock).toHaveBeenCalled();});- it("runs a read-only tool inside CLS without a transaction", () => {- createMcpServer();+ it("runs a read-only tool inside CLS without a transaction", async () => {+ await createMcpServer();const read = registered.find((r) => r.name === "read_tool")!;const result = read.callback({ foo: 1 });@@ -89,8 +92,8 @@ describe("createMcpServer", () => {expect(result).toEqual({ content: [{ type: "text", text: JSON.stringify({ ok: "read" }) }] });});- it("runs a mutating tool inside CLS + a transaction", () => {- createMcpServer();+ it("runs a mutating tool inside CLS + a transaction", async () => {+ await createMcpServer();const mutate = registered.find((r) => r.name === "mutate_tool")!;const result = mutate.callback({ bar: 2 });diff --git a/apps/server/src/services/mcp/mcp_server.ts b/apps/server/src/services/mcp/mcp_server.tsindex c87eeee78c..0b40f20ed0 100644--- a/apps/server/src/services/mcp/mcp_server.ts+++ b/apps/server/src/services/mcp/mcp_server.ts@@ -10,7 +10,7 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";import { app_info as appInfo } from "@triliumnext/core";import { cls } from "@triliumnext/core";-import { allToolRegistries } from "@triliumnext/core/src/services/llm/tools/index.js";+import { allToolRegistries, resolveToolRegistries } from "@triliumnext/core/src/services/llm/tools/index.js";import type { ToolDefinition } from "@triliumnext/core/src/services/llm/tools/tool_registry.js";import sql from "../sql.js";@@ -38,7 +38,11 @@ function registerTool(server: McpServer, name: string, def: ToolDefinition) {});}-export function createMcpServer(): McpServer {+export async function createMcpServer(): Promise<McpServer> {+ // Host-registered registries (the User Guide tools) load lazily; the array+ // is complete only after this resolves.+ await resolveToolRegistries();+const server = new McpServer({name: "trilium-notes",version: appInfo.appVersiondiff --git a/packages/trilium-core/src/services/llm/chat.ts b/packages/trilium-core/src/services/llm/chat.tsindex 9cbd814caa..499e127bc7 100644--- a/packages/trilium-core/src/services/llm/chat.ts+++ b/packages/trilium-core/src/services/llm/chat.ts@@ -14,6 +14,7 @@ import { safeExtractMessageAndStackFromError } from "../utils/index.js";import { generateChatTitle } from "./chat_title.js";import { getProvider, getProviderByType, getSelectedModel, hasConfiguredProviders, type LlmProviderConfig } from "./index.js";import { formatStreamError, streamToChunks } from "./stream.js";+import { resolveToolRegistries } from "./tools/index.js";/*** Stream one assistant turn, yielding the chunks as the provider produces them.@@ -40,9 +41,13 @@ export async function* runChat(// Prefer routing by the provider config id — it disambiguates multiple// configs of the same type (e.g. OpenAI + a self-hosted Ollama). Chats// saved before providerId existed fall back to type-based resolution.+ // Host-provided tool registries load lazily; the providers' synchronous+ // buildTools() reads the registry array, so it must be complete first.+ await resolveToolRegistries();+const provider = config.providerId- ? getProvider(config.providerId)- : getProviderByType(config.provider || "anthropic");+ ? await getProvider(config.providerId)+ : await getProviderByType(config.provider || "anthropic");const modelId = config.model || provider.getAvailableModels().find(m => m.isDefault)?.id;if (!modelId) {diff --git a/packages/trilium-core/src/services/llm/chat_title.ts b/packages/trilium-core/src/services/llm/chat_title.tsindex 7df6de3cee..64b2c1a038 100644--- a/packages/trilium-core/src/services/llm/chat_title.ts+++ b/packages/trilium-core/src/services/llm/chat_title.ts@@ -37,7 +37,7 @@ export async function generateChatTitle(chatNoteId: string, firstMessage: string// Whichever provider is configured first, which is not necessarily the one// the chat itself is talking to.- const provider = getProvider();+ const provider = await getProvider();log.info(`Naming chat note ${chatNoteId} with the ${provider.name} provider.`);const title = await provider.generateTitle(firstMessage);diff --git a/packages/trilium-core/src/services/llm/host_providers.ts b/packages/trilium-core/src/services/llm/host_providers.tsnew file mode 100644index 0000000000..452e7645a1--- /dev/null+++ b/packages/trilium-core/src/services/llm/host_providers.ts@@ -0,0 +1,74 @@+/**+ * The host-provider registration seam, in a module of its own so hosts can+ * register at startup without loading the LLM stack: `index.js` pulls every+ * SDK-backed provider on import, while this file needs only a type. Keeping+ * registration light is what lets the whole stack stay in lazy chunks until+ * the first chat or MCP request (see the ESM `splitting` in build-utils).+ */++import type { LlmProvider } from "./types.js";++/**+ * Provider types core cannot build for itself, each mapped to the name the user+ * knows it by.+ *+ * What they have in common is a dependency on the runtime around them rather+ * than on an API key: both are CLIs this process spawns, authenticating through+ * the host's own account plumbing rather than a key the user pastes in. Core+ * runs in the browser too, where none of that exists, so the host that *can* do+ * it registers a factory at startup with {@link registerHostProvider}. Where+ * nothing registers one, selecting the provider fails with a message naming it+ * rather than a bare "unknown type".+ *+ * Adding one means an entry here, a literal branch in `createProviderInstance`+ * (see the note there on why it must be literal), and a registration in+ * whichever hosts can serve it.+ */+export const HOST_PROVIDED_TYPES = {+ /** Claude Pro/Max through the Claude Agent SDK, authenticated by `claude /login`. */+ "claude-agent": "Claude Code",+ /** GitHub Copilot through the Copilot CLI's ACP mode, authenticated by `copilot login`. */+ "copilot-agent": "GitHub Copilot"+} as const;++/** A provider type from {@link HOST_PROVIDED_TYPES}. */+export type HostProvidedType = keyof typeof HOST_PROVIDED_TYPES;++/**+ * A factory can resolve asynchronously so the host can dynamically import its+ * provider module inside it — registration then costs nothing at startup.+ */+type HostProviderFactory = () => LlmProvider | Promise<LlmProvider>;++const hostProviderFactories = new Map<string, HostProviderFactory>();++/**+ * Register the host's implementation of a provider core cannot construct.+ * Called once at startup, before any chat can ask for it.+ */+export function registerHostProvider(type: HostProvidedType, factory: HostProviderFactory) {+ hostProviderFactories.set(type, factory);+}++/**+ * Forget every registered factory, putting the registry back to how a build that+ * supplies none starts. For a runtime that re-initialises core (tests, an+ * Electron reload), which registers again on the way back up.+ */+export function clearHostProviders() {+ hostProviderFactories.clear();+}++/**+ * Build a host-provided provider, or explain that this build has no one to build+ * it. The type is always a literal from the call site — see+ * `createProviderInstance` in `index.js` — never the string the user's config+ * carried.+ */+export async function createHostProvider(type: HostProvidedType): Promise<LlmProvider> {+ const factory = hostProviderFactories.get(type);+ if (!factory) {+ throw new Error(`The ${HOST_PROVIDED_TYPES[type]} provider is not available in this build.`);+ }+ return await factory();+}diff --git a/packages/trilium-core/src/services/llm/index.spec.ts b/packages/trilium-core/src/services/llm/index.spec.tsindex 772e40018d..3673bcf514 100644--- a/packages/trilium-core/src/services/llm/index.spec.ts+++ b/packages/trilium-core/src/services/llm/index.spec.ts@@ -91,21 +91,21 @@ describe("llm/index provider registry", () => {});describe("getProvider", () => {- it("returns the first provider when no id is given and caches it", () => {+ it("returns the first provider when no id is given and caches it", async () => {setProviders(TWO);- const p1 = getProvider();- const p2 = getProvider();+ const p1 = await getProvider();+ const p2 = await getProvider();expect(p1).toBe(p2); // cachedexpect((p1.constructor as any).lastArgs).toEqual(["k1", "https://proxy"]);});- it("returns the provider matching a given id", () => {+ it("returns the provider matching a given id", async () => {setProviders(TWO);- const p = getProvider("o1");+ const p = await getProvider("o1");expect((p.constructor as any).lastArgs).toEqual(["k2", undefined]);});- it("instantiates each known provider type via its factory", () => {+ it("instantiates each known provider type via its factory", async () => {setProviders([{ id: "a", name: "A", provider: "anthropic", apiKey: "ka" },{ id: "o", name: "O", provider: "openai", apiKey: "ko" },@@ -116,68 +116,69 @@ describe("llm/index provider registry", () => {{ id: "lm", name: "LM", provider: "lmstudio", apiKey: "", baseURL: "http://box:1234/v1" },{ id: "oc", name: "OC", provider: "openai-compatible", apiKey: "k", baseURL: "http://box:8080/v1" }]);- expect((getProvider("a").constructor as any).lastArgs).toEqual(["ka", undefined]);- expect((getProvider("o").constructor as any).lastArgs).toEqual(["ko", undefined]);- expect((getProvider("g").constructor as any).lastArgs).toEqual(["kg", undefined]);+ const constructorArgs = async (id: string) => ((await getProvider(id)).constructor as any).lastArgs;+ expect(await constructorArgs("a")).toEqual(["ka", undefined]);+ expect(await constructorArgs("o")).toEqual(["ko", undefined]);+ expect(await constructorArgs("g")).toEqual(["kg", undefined]);// Its own class rather than the shared self-hosted one, despite speaking// the same protocol — that is what gives its models a price.- expect((getProvider("d").constructor as any).lastArgs).toEqual(["kd", undefined]);+ expect(await constructorArgs("d")).toEqual(["kd", undefined]);// The subscription provider takes no constructor args — auth is Claude Code's.- expect((getProvider("c").constructor as any).lastArgs).toEqual([]);+ expect(await constructorArgs("c")).toEqual([]);// The three self-hosted cards share one class, which receives the card// id so it knows which endpoint to probe and prefill.- expect((getProvider("l").constructor as any).lastArgs).toEqual(["ollama", "", "http://ollama.lan:11434"]);- expect((getProvider("lm").constructor as any).lastArgs).toEqual(["lmstudio", "", "http://box:1234/v1"]);- expect((getProvider("oc").constructor as any).lastArgs).toEqual(["openai-compatible", "k", "http://box:8080/v1"]);+ expect(await constructorArgs("l")).toEqual(["ollama", "", "http://ollama.lan:11434"]);+ expect(await constructorArgs("lm")).toEqual(["lmstudio", "", "http://box:1234/v1"]);+ expect(await constructorArgs("oc")).toEqual(["openai-compatible", "k", "http://box:8080/v1"]);});- it("throws when no providers are configured (null and empty array)", () => {+ it("throws when no providers are configured (null and empty array)", async () => {getOptionOrNullMock.mockReturnValue(null);- expect(() => getProvider()).toThrow(/No LLM providers configured/);+ await expect(getProvider()).rejects.toThrow(/No LLM providers configured/);setProviders([]);- expect(() => getProvider()).toThrow(/No LLM providers configured/);+ await expect(getProvider()).rejects.toThrow(/No LLM providers configured/);});- it("throws when the requested id is not found", () => {+ it("throws when the requested id is not found", async () => {setProviders(TWO);- expect(() => getProvider("nope")).toThrow(/not found: nope/);+ await expect(getProvider("nope")).rejects.toThrow(/not found: nope/);});- it("throws for an unknown provider type", () => {+ it("throws for an unknown provider type", async () => {setProviders([{ id: "x", name: "X", provider: "mystery", apiKey: "k" }]);- expect(() => getProvider("x")).toThrow(/Unknown LLM provider type: mystery/);+ await expect(getProvider("x")).rejects.toThrow(/Unknown LLM provider type: mystery/);});- it("builds every host-provided type through the factory its host registered", () => {+ it("builds every host-provided type through the factory its host registered", async () => {// The catalog and the switch have to agree: a type listed in// HOST_PROVIDED_TYPES with no branch of its own falls through to the// default and reads as a typo, in a build that can in fact serve it.const types = Object.keys(HOST_PROVIDED_TYPES);setProviders(types.map((provider, index) => ({ id: `h${index}`, name: provider, provider, apiKey: "" })));- types.forEach((type, index) => {- expect(getProvider(`h${index}`)).toBeInstanceOf(hostProviderMocks[type]);- });+ for (const [index, type] of types.entries()) {+ expect(await getProvider(`h${index}`)).toBeInstanceOf(hostProviderMocks[type]);+ }});- it("names the provider when no host in this build can build it", () => {+ it("names the provider when no host in this build can build it", async () => {// What standalone hits for Claude Code: the type is real and the branch is// there, but nothing registered a factory. Better than "unknown type",// which would send the user looking for a typo.clearHostProviders();setProviders([{ id: "c", name: "C", provider: "claude-agent", apiKey: "" }]);- expect(() => getProvider("c")).toThrow("The Claude Code provider is not available in this build.");+ await expect(getProvider("c")).rejects.toThrow("The Claude Code provider is not available in this build.");});- it("drops cached instances when the llmProviders option changes", () => {+ it("drops cached instances when the llmProviders option changes", async () => {setProviders(TWO);- const p1 = getProvider("a1");+ const p1 = await getProvider("a1");// Same config → still cached.setProviders(TWO);- expect(getProvider("a1")).toBe(p1);+ expect(await getProvider("a1")).toBe(p1);// Edited config (e.g. new API key) → cache self-invalidates.setProviders([{ ...TWO[0], apiKey: "new-key" }, TWO[1]]);- const p2 = getProvider("a1");+ const p2 = await getProvider("a1");expect(p2).not.toBe(p1);expect((p2.constructor as any).lastArgs).toEqual(["new-key", "https://proxy"]);});@@ -202,15 +203,15 @@ describe("llm/index provider registry", () => {});describe("getProviderByType", () => {- it("returns the first provider of the given type", () => {+ it("returns the first provider of the given type", async () => {setProviders(TWO);- const p = getProviderByType("openai");+ const p = await getProviderByType("openai");expect((p.constructor as any).lastArgs).toEqual(["k2", undefined]);});- it("throws when no provider of that type is configured", () => {+ it("throws when no provider of that type is configured", async () => {setProviders(TWO);- expect(() => getProviderByType("google")).toThrow(/No google provider configured/);+ await expect(getProviderByType("google")).rejects.toThrow(/No google provider configured/);});});diff --git a/packages/trilium-core/src/services/llm/index.ts b/packages/trilium-core/src/services/llm/index.tsindex 535ce8318c..fef3eb46ad 100644--- a/packages/trilium-core/src/services/llm/index.ts+++ b/packages/trilium-core/src/services/llm/index.ts@@ -1,13 +1,11 @@import { getLog } from "../../services/log.js";import optionService from "../../services/options.js";-import { AnthropicProvider } from "./providers/anthropic.js";-import { DeepSeekProvider } from "./providers/deepseek.js";-import { GoogleProvider } from "./providers/google.js";-import { LocalProvider } from "./providers/local.js";-import { OpenAiProvider } from "./providers/openai.js";+import { createHostProvider } from "./host_providers.js";import type { LlmProvider, ModelInfo } from "./types.js";+export { clearHostProviders, HOST_PROVIDED_TYPES, type HostProvidedType, registerHostProvider } from "./host_providers.js";+/*** Configuration for a single LLM provider instance.* This matches the structure stored in the llmProviders option.@@ -30,64 +28,6 @@ export interface LlmProviderSetup {/** Provider type identifiers that can be instantiated, for error messages. */const PROVIDER_TYPES = ["anthropic", "openai", "google", "deepseek", "claude-agent", "copilot-agent", "ollama", "lmstudio", "openai-compatible"];-/**- * Provider types core cannot build for itself, each mapped to the name the user- * knows it by.- *- * What they have in common is a dependency on the runtime around them rather- * than on an API key: both are CLIs this process spawns, authenticating through- * the host's own account plumbing rather than a key the user pastes in. Core- * runs in the browser too, where none of that exists, so the host that *can* do- * it registers a factory at startup with {@link registerHostProvider}. Where- * nothing registers one, selecting the provider fails with a message naming it- * rather than a bare "unknown type".- *- * Adding one means an entry here, a literal branch in- * {@link createProviderInstance} (see the note there on why it must be literal),- * and a registration in whichever hosts can serve it.- */-export const HOST_PROVIDED_TYPES = {- /** Claude Pro/Max through the Claude Agent SDK, authenticated by `claude /login`. */- "claude-agent": "Claude Code",- /** GitHub Copilot through the Copilot CLI's ACP mode, authenticated by `copilot login`. */- "copilot-agent": "GitHub Copilot"-} as const;--/** A provider type from {@link HOST_PROVIDED_TYPES}. */-export type HostProvidedType = keyof typeof HOST_PROVIDED_TYPES;--const hostProviderFactories = new Map<string, () => LlmProvider>();--/**- * Register the host's implementation of a provider core cannot construct.- * Called once at startup, before any chat can ask for it.- */-export function registerHostProvider(type: HostProvidedType, factory: () => LlmProvider) {- hostProviderFactories.set(type, factory);-}--/**- * Forget every registered factory, putting the registry back to how a build that- * supplies none starts. For a runtime that re-initialises core (tests, an- * Electron reload), which registers again on the way back up.- */-export function clearHostProviders() {- hostProviderFactories.clear();-}--/**- * Build a host-provided provider, or explain that this build has no one to build- * it. The type is always a literal from the call site — see- * {@link createProviderInstance} — never the string the user's config carried.- */-function createHostProvider(type: HostProvidedType): LlmProvider {- const factory = hostProviderFactories.get(type);- if (!factory) {- throw new Error(`The ${HOST_PROVIDED_TYPES[type]} provider is not available in this build.`);- }- return factory();-}-/*** Instantiate a provider from its type identifier.*@@ -100,33 +40,38 @@ function createHostProvider(type: HostProvidedType): LlmProvider {* The host-provided factories are looked up too, but only ever under a literal* spelled out in their branch below — the untrusted string picks the branch and* goes no further.+ *+ * Each branch imports its provider module on first use: the SDK behind it must+ * stay out of the startup path (and out of the standalone worker's boot) for+ * users who never chat, so the import specifiers are literals inside the+ * branches rather than at module scope.*/-function createProviderInstance(provider: string, apiKey: string, baseURL?: string): LlmProvider {+async function createProviderInstance(provider: string, apiKey: string, baseURL?: string): Promise<LlmProvider> {switch (provider) {case "anthropic":- return new AnthropicProvider(apiKey, baseURL);+ return new (await import("./providers/anthropic.js")).AnthropicProvider(apiKey, baseURL);case "openai":- return new OpenAiProvider(apiKey, baseURL);+ return new (await import("./providers/openai.js")).OpenAiProvider(apiKey, baseURL);case "google":- return new GoogleProvider(apiKey, baseURL);+ return new (await import("./providers/google.js")).GoogleProvider(apiKey, baseURL);// OpenAI-compatible on the wire, but carded separately from the generic// custom endpoint so its models resolve against the committed price table.case "deepseek":- return new DeepSeekProvider(apiKey, baseURL);+ return new (await import("./providers/deepseek.js")).DeepSeekProvider(apiKey, baseURL);// Subscription providers, whose credentials belong to the host rather than// to a key the user pastes in — so the host builds them. One branch each,// naming its own type: see HOST_PROVIDED_TYPES.case "claude-agent":- return createHostProvider("claude-agent");+ return await createHostProvider("claude-agent");case "copilot-agent":- return createHostProvider("copilot-agent");+ return await createHostProvider("copilot-agent");// Self-hosted endpoints. The three cards differ only in the URL and setup// hint the UI prefills; they all speak the OpenAI-compatible API, with// Ollama and LM Studio additionally offering a richer native listing.case "ollama":case "lmstudio":case "openai-compatible":- return new LocalProvider(provider, apiKey, baseURL);+ return new (await import("./providers/local.js")).LocalProvider(provider, apiKey, baseURL);default:throw new Error(`Unknown LLM provider type: ${provider}. Available: ${PROVIDER_TYPES.join(", ")}`);}@@ -166,7 +111,7 @@ function getConfiguredProviders(): LlmProviderSetup[] {* Get a provider instance by its configuration ID.* If no ID is provided, returns the first configured provider.*/-export function getProvider(providerId?: string): LlmProvider {+export async function getProvider(providerId?: string): Promise<LlmProvider> {const configs = getConfiguredProviders();if (configs.length === 0) {@@ -188,7 +133,7 @@ export function getProvider(providerId?: string): LlmProvider {}// Create new provider instance- const provider = createProviderInstance(config.provider, config.apiKey, config.baseURL);+ const provider = await createProviderInstance(config.provider, config.apiKey, config.baseURL);cachedProviders[config.id] = provider;return provider;}@@ -196,7 +141,7 @@ export function getProvider(providerId?: string): LlmProvider {/*** Get the first configured provider of a specific type (e.g., "anthropic").*/-export function getProviderByType(providerType: string): LlmProvider {+export async function getProviderByType(providerType: string): Promise<LlmProvider> {const configs = getConfiguredProviders();const config = configs.find(c => c.provider === providerType);@@ -224,7 +169,7 @@ export function hasConfiguredProviders(): boolean {* dynamic listing.*/export async function listProviderModels(provider: string, apiKey: string, baseURL?: string): Promise<ModelInfo[]> {- const instance = createProviderInstance(provider, apiKey, baseURL);+ const instance = await createProviderInstance(provider, apiKey, baseURL);const models = await (instance.listModels?.() ?? instance.getAvailableModels());// Tag the default-selected set here so the recommendation rule lives on the// server — in the provider that owns its model id shape — rather than in thediff --git a/packages/trilium-core/src/services/llm/tools/index.ts b/packages/trilium-core/src/services/llm/tools/index.tsindex 15b56af2d3..765bf6d4f8 100644--- a/packages/trilium-core/src/services/llm/tools/index.ts+++ b/packages/trilium-core/src/services/llm/tools/index.ts@@ -17,6 +17,7 @@ import { attributeTools } from "./attribute_tools.js";import { hierarchyTools } from "./hierarchy_tools.js";import { iconTools } from "./icon_tools.js";import { noteTools } from "./note_tools.js";+import { drainToolRegistryLoaders } from "./registration.js";import { skillTools } from "./skill_tools.js";import type { ToolRegistry } from "./tool_registry.js";@@ -44,9 +45,25 @@ export const allToolRegistries: ToolRegistry[] = [* or MCP tool listing, both of which read {@link allToolRegistries} eagerly.* Registering the same registry twice is a no-op, so a host that initialises* more than once (tests, Electron reloads) does not duplicate its tools.+ *+ * A host whose registry should stay out of the startup path registers a loader+ * with `registerToolRegistryLoader` (see `registration.js`) instead of calling+ * this directly.*/export function registerToolRegistry(registry: ToolRegistry) {if (!allToolRegistries.includes(registry)) {allToolRegistries.push(registry);}}++/**+ * Loads every host-registered lazy registry into {@link allToolRegistries}.+ * The chat pipeline and the MCP server call this at the start of their async+ * entry points, which is what lets `buildTools()` and other readers stay+ * synchronous: by the time they iterate the array, it is complete.+ */+export async function resolveToolRegistries(): Promise<void> {+ for (const loader of drainToolRegistryLoaders()) {+ registerToolRegistry(await loader());+ }+}diff --git a/packages/trilium-core/src/services/llm/tools/registration.ts b/packages/trilium-core/src/services/llm/tools/registration.tsnew file mode 100644index 0000000000..d18d812249--- /dev/null+++ b/packages/trilium-core/src/services/llm/tools/registration.ts@@ -0,0 +1,30 @@+/**+ * The lazy half of the tool-registry seam, in a module of its own so hosts can+ * register at startup without loading the tool stack: `index.js` pulls every+ * core registry (and through `tool_registry.js` the AI SDK and zod) on import,+ * while this file needs only a type. A host registers a loader here; the first+ * chat turn or MCP request resolves it via `resolveToolRegistries()` in+ * `index.js`, and the registry's module loads then, not at startup.+ */++import type { ToolRegistry } from "./tool_registry.js";++/** Imports and returns a host's tool registry on first use. */+export type ToolRegistryLoader = () => Promise<ToolRegistry>;++const pendingLoaders: ToolRegistryLoader[] = [];++/**+ * Register a loader for a host-provided tool registry. Called once at startup;+ * registering again after a resolve (tests, an Electron reload) queues the+ * loader again, and the duplicate collapses when it resolves to the same+ * registry instance.+ */+export function registerToolRegistryLoader(loader: ToolRegistryLoader) {+ pendingLoaders.push(loader);+}++/** Hands the queued loaders to `resolveToolRegistries()`, emptying the queue. */+export function drainToolRegistryLoaders(): ToolRegistryLoader[] {+ return pendingLoaders.splice(0, pendingLoaders.length);+}