Skip to content

Build your personal knowledge base with Trilium Notes

perf(llm): keep the LLM stack out of the startup path

Elian Doranb4bbc501799d
diff --git a/.claude/skills/adding-llm-mcp-tools/SKILL.md b/.claude/skills/adding-llm-mcp-tools/SKILL.md
index ca85d3549b..e54acd8b36 100644
--- a/.claude/skills/adding-llm-mcp-tools/SKILL.md
+++ b/.claude/skills/adding-llm-mcp-tools/SKILL.md
@@ -36,9 +36,12 @@ Rule: **any tool that writes** (`setContent`, `save`, `setAttribute`, `createNew
**Node-only tools take the other door.** A tool that needs something core cannot have in the browser
(the in-app documentation reader, for instance) lives in `apps/server/src/services/llm/tools/`
-(`doc_notes.ts`, `help_tools.ts`) and registers itself at server startup by calling
-`registerToolRegistry(theRegistry)` (`index.ts:48`), which appends to the same array. Standalone
-simply runs without it. Put a tool there **only** if it genuinely cannot run under sqlite-wasm —
+(`doc_notes.ts`, `help_tools.ts`) and registers at server startup via
+`registerToolRegistryLoader(async () => (await import("./tools/x.js")).xTools)` from core's light
+`tools/registration.ts` (see `registerServerLlmExtensions`); the first chat turn or MCP request
+resolves the loader through `resolveToolRegistries()` and appends the registry to the same array.
+Register the loader, never a static import — the loader form is what keeps the tool stack (zod, the
+AI SDK) out of the server's startup path. Standalone simply runs without it. Put a tool there **only** if it genuinely cannot run under sqlite-wasm —
core is the default home, and anything placed in the server loses the standalone and desktop
consumers.
diff --git a/CLAUDE.md b/CLAUDE.md
index 0652bd09df..5b03a075a2 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -232,7 +232,7 @@ The `_hidden` subtree holds system notes with deterministic `_`-prefixed IDs so
- Operations called from Express routes already have CLS context; standalone services (schedulers, Electron IPC handlers) do not
### Adding New LLM Tools
-Tools are `defineTools({...})` registries in `packages/trilium-core/src/services/llm/tools/` (`note_tools`, `attribute_tools`, `attachment_tools`, `hierarchy_tools`, …), served to both the LLM chat and the MCP server. Each tool: `description`, Zod `inputSchema`, `execute`, `mutates: true` for writes. A new module goes into `allToolRegistries` in `tools/index.ts`; Node-only tools (in-app docs) live in `apps/server/src/services/llm/tools/` and use `registerToolRegistry()` from `apps/server/src/services/llm/index.ts`. Add the client label under `llm.tools.<tool_name>` in imperative tense ("Search notes"). Mirror ETAPI's field choices but **don't import ETAPI mappers** — inline them so the LLM layer stays decoupled.
+Tools are `defineTools({...})` registries in `packages/trilium-core/src/services/llm/tools/` (`note_tools`, `attribute_tools`, `attachment_tools`, `hierarchy_tools`, …), served to both the LLM chat and the MCP server. Each tool: `description`, Zod `inputSchema`, `execute`, `mutates: true` for writes. A new module goes into `allToolRegistries` in `tools/index.ts`; Node-only tools (in-app docs) live in `apps/server/src/services/llm/tools/` and register via `registerToolRegistryLoader()` (a lazy dynamic-import loader — see `registerServerLlmExtensions` in `apps/server/src/services/llm/index.ts`) so the tool stack stays out of the startup path. Add the client label under `llm.tools.<tool_name>` in imperative tense ("Search notes"). Mirror ETAPI's field choices but **don't import ETAPI mappers** — inline them so the LLM layer stays decoupled.
### Server-Side Static Assets
Node-side assets (templates, translations, prompts) live in `apps/server/src/assets/` and are read via `RESOURCE_DIR` from `apps/server/src/services/resource_dir.ts` (`path.join(RESOURCE_DIR, "llm", "prompts", …)`); assets core itself reads (`schema.sql`, LLM skills) live in `packages/trilium-core/src/assets/`. **Never resolve paths with `import.meta.url`/`fileURLToPath` or `__dirname` + relative path** — the server is bundled to CJS and both point at the bundle, not the source tree.
diff --git a/apps/server/src/routes/api/llm_chat.ts b/apps/server/src/routes/api/llm_chat.ts
index b4937a6880..929c733b13 100644
--- a/apps/server/src/routes/api/llm_chat.ts
+++ b/apps/server/src/routes/api/llm_chat.ts
@@ -1,5 +1,4 @@
import type { LlmMessage } from "@triliumnext/commons";
-import { runChat } from "@triliumnext/core/src/services/llm/chat.js";
import type { LlmProviderConfig } from "@triliumnext/core/src/services/llm/types.js";
import type { Request, Response } from "express";
@@ -46,6 +45,10 @@ async function streamChat(req: Request, res: Response) {
res.on("close", () => abortController.abort());
try {
+ // Imported here rather than at module scope so the chat pipeline and
+ // the provider SDKs land in lazy chunks (the same convention as
+ // getProviderModels in core's routes/api/llm.ts).
+ const { runChat } = await import("@triliumnext/core/src/services/llm/chat.js");
for await (const chunk of runChat(messages, config, abortController.signal)) {
res.write(`data: ${JSON.stringify(chunk)}\n\n`);
// Flush immediately to ensure real-time streaming
diff --git a/apps/server/src/routes/mcp.ts b/apps/server/src/routes/mcp.ts
index 5084626e03..1ea9f1f76e 100644
--- a/apps/server/src/routes/mcp.ts
+++ b/apps/server/src/routes/mcp.ts
@@ -8,13 +8,11 @@
* arrangement outright rather than layering on top of it.
*/
-import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { getLog, options as optionService } from "@triliumnext/core";
import type express from "express";
import rateLimit from "express-rate-limit";
import etapiTokenService from "../services/etapi_tokens.js";
-import { createMcpServer } from "../services/mcp/mcp_server.js";
export function register(app: express.Application) {
const mcpRateLimiter = createMcpRateLimiter();
@@ -124,7 +122,15 @@ function mcpGuard(req: express.Request, res: express.Response, next: express.Nex
async function handleMcpRequest(req: express.Request, res: express.Response) {
try {
- const server = createMcpServer();
+ // Imported here rather than at module scope so the MCP SDK and the
+ // tool stack land in lazy chunks: the route registers at startup, but
+ // the guard above answers unauthenticated traffic before any of this
+ // loads.
+ const [{ StreamableHTTPServerTransport }, { createMcpServer }] = await Promise.all([
+ import("@modelcontextprotocol/sdk/server/streamableHttp.js"),
+ import("../services/mcp/mcp_server.js")
+ ]);
+ const server = await createMcpServer();
const transport = new StreamableHTTPServerTransport({
sessionIdGenerator: undefined, // stateless
// Off because the guard above has already required a bearer token, which the
diff --git a/apps/server/src/services/llm/index.spec.ts b/apps/server/src/services/llm/index.spec.ts
index 8f5aa7ce20..e63c8ee4b5 100644
--- a/apps/server/src/services/llm/index.spec.ts
+++ b/apps/server/src/services/llm/index.spec.ts
@@ -4,20 +4,22 @@ const registrations = vi.hoisted(() => ({
hostProviders: [] as { type: string; factory: () => unknown }[],
docNoteReader: undefined as ((note: unknown) => string | null) | undefined,
skillReader: undefined as ((file: string) => string | null) | undefined,
- toolRegistries: [] as unknown[]
+ toolRegistryLoaders: [] as (() => Promise<unknown>)[]
}));
-vi.mock("@triliumnext/core/src/services/llm/index.js", () => ({
+vi.mock("@triliumnext/core/src/services/llm/host_providers.js", () => ({
registerHostProvider: (type: string, factory: () => unknown) => { registrations.hostProviders.push({ type, factory }); }
}));
-vi.mock("@triliumnext/core/src/services/llm/skills.js", () => ({
+vi.mock("@triliumnext/core/src/services/llm/skills.js", async (importOriginal) => ({
+ ...await importOriginal<typeof import("@triliumnext/core/src/services/llm/skills.js")>(),
registerSkillReader: (reader: (file: string) => string | null) => { registrations.skillReader = reader; }
}));
-vi.mock("@triliumnext/core/src/services/llm/tools/helpers.js", () => ({
+vi.mock("@triliumnext/core/src/services/llm/tools/helpers.js", async (importOriginal) => ({
+ ...await importOriginal<typeof import("@triliumnext/core/src/services/llm/tools/helpers.js")>(),
registerDocNoteHtmlReader: (reader: (note: unknown) => string | null) => { registrations.docNoteReader = reader; }
}));
-vi.mock("@triliumnext/core/src/services/llm/tools/index.js", () => ({
- registerToolRegistry: (registry: unknown) => { registrations.toolRegistries.push(registry); }
+vi.mock("@triliumnext/core/src/services/llm/tools/registration.js", () => ({
+ registerToolRegistryLoader: (loader: () => Promise<unknown>) => { registrations.toolRegistryLoaders.push(loader); }
}));
import { registerServerLlmExtensions } from "./index.js";
@@ -26,21 +28,24 @@ import { CopilotAgentProvider } from "./providers/copilot_agent.js";
import { helpTools } from "./tools/help_tools.js";
describe("registerServerLlmExtensions", () => {
- it("hands core every piece of the stack that needs Node", () => {
+ it("hands core every piece of the stack that needs Node", async () => {
registerServerLlmExtensions();
// The subscription providers, which shell out to the Claude Code and Copilot
- // CLIs, under the types core knows them by. Registered as factories, so
- // nothing is constructed until a chat asks for it.
+ // CLIs, under the types core knows them by. Registered as async factories
+ // that import their module on first use, so neither the provider nor its
+ // agent SDK loads until a chat asks for it.
expect(registrations.hostProviders.map(p => p.type)).toEqual(["claude-agent", "copilot-agent"]);
- expect(registrations.hostProviders[0].factory()).toBeInstanceOf(ClaudeAgentProvider);
- expect(registrations.hostProviders[1].factory()).toBeInstanceOf(CopilotAgentProvider);
+ expect(await registrations.hostProviders[0].factory()).toBeInstanceOf(ClaudeAgentProvider);
+ expect(await registrations.hostProviders[1].factory()).toBeInstanceOf(CopilotAgentProvider);
// The User Guide reader, which core's note-content helper calls for doc notes.
expect(registrations.docNoteReader).toBeTypeOf("function");
// The skill sheets: core owns the catalog and the tool, the server only
// knows how to get a sheet off disk.
expect(registrations.skillReader?.("search_syntax.md")).toContain("#");
- // The one registry still backed by files this build alone carries.
- expect(registrations.toolRegistries).toEqual([helpTools]);
+ // The one registry still backed by files this build alone carries,
+ // registered as a loader so the tool stack stays out of the startup path.
+ expect(registrations.toolRegistryLoaders).toHaveLength(1);
+ expect(await registrations.toolRegistryLoaders[0]()).toBe(helpTools);
});
});
diff --git a/apps/server/src/services/llm/index.ts b/apps/server/src/services/llm/index.ts
index 451c0e52e8..4f0a9d9592 100644
--- a/apps/server/src/services/llm/index.ts
+++ b/apps/server/src/services/llm/index.ts
@@ -12,27 +12,29 @@
* one more line here and one more entry in core's HOST_PROVIDED_TYPES.
*/
-import { registerHostProvider } from "@triliumnext/core/src/services/llm/index.js";
+import { registerHostProvider } from "@triliumnext/core/src/services/llm/host_providers.js";
import { registerSkillReader } from "@triliumnext/core/src/services/llm/skills.js";
import { registerDocNoteHtmlReader } from "@triliumnext/core/src/services/llm/tools/helpers.js";
-import { registerToolRegistry } from "@triliumnext/core/src/services/llm/tools/index.js";
+import { registerToolRegistryLoader } from "@triliumnext/core/src/services/llm/tools/registration.js";
import { loadSkillSheet } from "../../core_assets.js";
-import { ClaudeAgentProvider } from "./providers/claude_agent.js";
-import { CopilotAgentProvider } from "./providers/copilot_agent.js";
import { getDocNoteHtml } from "./tools/doc_notes.js";
-import { helpTools } from "./tools/help_tools.js";
/**
* Contribute those pieces to core. Called once from startup, beside the other
* registrations there, rather than run as an import side effect: core's own LLM
* routes reach the stack directly, so what a chat can use must not depend on
* whether some module that happens to import this one was loaded first.
+ *
+ * The providers and the help tools are registered as loaders, not values: their
+ * modules carry the agent SDKs and the tool stack, and importing them here
+ * would put all of that in the startup path of every install. The registration
+ * seams are light on purpose — core resolves each loader on first use.
*/
export function registerServerLlmExtensions() {
- registerHostProvider("claude-agent", () => new ClaudeAgentProvider());
- registerHostProvider("copilot-agent", () => new CopilotAgentProvider());
+ registerHostProvider("claude-agent", async () => new (await import("./providers/claude_agent.js")).ClaudeAgentProvider());
+ registerHostProvider("copilot-agent", async () => new (await import("./providers/copilot_agent.js")).CopilotAgentProvider());
registerDocNoteHtmlReader(getDocNoteHtml);
registerSkillReader(loadSkillSheet);
- registerToolRegistry(helpTools);
+ registerToolRegistryLoader(async () => (await import("./tools/help_tools.js")).helpTools);
}
diff --git a/apps/server/src/services/llm/providers/claude_agent.ts b/apps/server/src/services/llm/providers/claude_agent.ts
index 57136e6f01..7d1cba4c7d 100644
--- a/apps/server/src/services/llm/providers/claude_agent.ts
+++ b/apps/server/src/services/llm/providers/claude_agent.ts
@@ -634,7 +634,7 @@ export class ClaudeAgentProvider implements LlmProvider {
// open port, and no dependency on the user-facing `mcpEnabled`
// toggle (which exists to expose notes to *external* clients).
options.mcpServers = {
- trilium: { type: "sdk", name: "trilium", instance: createMcpServer() }
+ trilium: { type: "sdk", name: "trilium", instance: await createMcpServer() }
};
// Bare server prefix auto-allows every tool from that server.
allowedTools.push("mcp__trilium");
diff --git a/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts b/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts
index 66d54090c6..de0360baa8 100644
--- a/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts
+++ b/apps/server/src/services/llm/providers/copilot_mcp_endpoint.ts
@@ -105,7 +105,7 @@ async function handleRequest(
}
// Stateless per-request server+transport, mirroring the public /mcp route.
- const mcpServer = createMcpServer();
+ const mcpServer = await createMcpServer();
const transport = new StreamableHTTPServerTransport({
sessionIdGenerator: undefined,
enableDnsRebindingProtection: true,
diff --git a/apps/server/src/services/mcp/mcp_server.spec.ts b/apps/server/src/services/mcp/mcp_server.spec.ts
index 116569c9a2..4271a8c4a7 100644
--- a/apps/server/src/services/mcp/mcp_server.spec.ts
+++ b/apps/server/src/services/mcp/mcp_server.spec.ts
@@ -52,8 +52,10 @@ const mutateRegistry: any = [
["mutate_tool", { description: "m", inputSchema: {}, mutates: true, execute: mutateExecute }]
];
+const resolveToolRegistriesMock = vi.fn(async () => {});
vi.mock("@triliumnext/core/src/services/llm/tools/index.js", () => ({
- allToolRegistries: [readRegistry, mutateRegistry]
+ allToolRegistries: [readRegistry, mutateRegistry],
+ resolveToolRegistries: resolveToolRegistriesMock
}));
const { createMcpServer } = await import("./mcp_server.js");
@@ -69,16 +71,17 @@ beforeEach(() => {
afterEach(() => vi.restoreAllMocks());
describe("createMcpServer", () => {
- it("registers every tool from all registries", () => {
- const server = createMcpServer();
+ it("registers every tool from all registries, after resolving the lazy ones", async () => {
+ const server = await createMcpServer();
expect(server).toBeInstanceOf(FakeMcpServer);
expect((server as any).info).toMatchObject({ name: "trilium-notes", version: "9.9.9-test" });
expect(registered.map((r) => r.name)).toEqual(["read_tool", "mutate_tool"]);
+ expect(resolveToolRegistriesMock).toHaveBeenCalled();
});
- it("runs a read-only tool inside CLS without a transaction", () => {
- createMcpServer();
+ it("runs a read-only tool inside CLS without a transaction", async () => {
+ await createMcpServer();
const read = registered.find((r) => r.name === "read_tool")!;
const result = read.callback({ foo: 1 });
@@ -89,8 +92,8 @@ describe("createMcpServer", () => {
expect(result).toEqual({ content: [{ type: "text", text: JSON.stringify({ ok: "read" }) }] });
});
- it("runs a mutating tool inside CLS + a transaction", () => {
- createMcpServer();
+ it("runs a mutating tool inside CLS + a transaction", async () => {
+ await createMcpServer();
const mutate = registered.find((r) => r.name === "mutate_tool")!;
const result = mutate.callback({ bar: 2 });
diff --git a/apps/server/src/services/mcp/mcp_server.ts b/apps/server/src/services/mcp/mcp_server.ts
index c87eeee78c..0b40f20ed0 100644
--- a/apps/server/src/services/mcp/mcp_server.ts
+++ b/apps/server/src/services/mcp/mcp_server.ts
@@ -10,7 +10,7 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { app_info as appInfo } from "@triliumnext/core";
import { cls } from "@triliumnext/core";
-import { allToolRegistries } from "@triliumnext/core/src/services/llm/tools/index.js";
+import { allToolRegistries, resolveToolRegistries } from "@triliumnext/core/src/services/llm/tools/index.js";
import type { ToolDefinition } from "@triliumnext/core/src/services/llm/tools/tool_registry.js";
import sql from "../sql.js";
@@ -38,7 +38,11 @@ function registerTool(server: McpServer, name: string, def: ToolDefinition) {
});
}
-export function createMcpServer(): McpServer {
+export async function createMcpServer(): Promise<McpServer> {
+ // Host-registered registries (the User Guide tools) load lazily; the array
+ // is complete only after this resolves.
+ await resolveToolRegistries();
+
const server = new McpServer({
name: "trilium-notes",
version: appInfo.appVersion
diff --git a/packages/trilium-core/src/services/llm/chat.ts b/packages/trilium-core/src/services/llm/chat.ts
index 9cbd814caa..499e127bc7 100644
--- a/packages/trilium-core/src/services/llm/chat.ts
+++ b/packages/trilium-core/src/services/llm/chat.ts
@@ -14,6 +14,7 @@ import { safeExtractMessageAndStackFromError } from "../utils/index.js";
import { generateChatTitle } from "./chat_title.js";
import { getProvider, getProviderByType, getSelectedModel, hasConfiguredProviders, type LlmProviderConfig } from "./index.js";
import { formatStreamError, streamToChunks } from "./stream.js";
+import { resolveToolRegistries } from "./tools/index.js";
/**
* Stream one assistant turn, yielding the chunks as the provider produces them.
@@ -40,9 +41,13 @@ export async function* runChat(
// Prefer routing by the provider config id — it disambiguates multiple
// configs of the same type (e.g. OpenAI + a self-hosted Ollama). Chats
// saved before providerId existed fall back to type-based resolution.
+ // Host-provided tool registries load lazily; the providers' synchronous
+ // buildTools() reads the registry array, so it must be complete first.
+ await resolveToolRegistries();
+
const provider = config.providerId
- ? getProvider(config.providerId)
- : getProviderByType(config.provider || "anthropic");
+ ? await getProvider(config.providerId)
+ : await getProviderByType(config.provider || "anthropic");
const modelId = config.model || provider.getAvailableModels().find(m => m.isDefault)?.id;
if (!modelId) {
diff --git a/packages/trilium-core/src/services/llm/chat_title.ts b/packages/trilium-core/src/services/llm/chat_title.ts
index 7df6de3cee..64b2c1a038 100644
--- a/packages/trilium-core/src/services/llm/chat_title.ts
+++ b/packages/trilium-core/src/services/llm/chat_title.ts
@@ -37,7 +37,7 @@ export async function generateChatTitle(chatNoteId: string, firstMessage: string
// Whichever provider is configured first, which is not necessarily the one
// the chat itself is talking to.
- const provider = getProvider();
+ const provider = await getProvider();
log.info(`Naming chat note ${chatNoteId} with the ${provider.name} provider.`);
const title = await provider.generateTitle(firstMessage);
diff --git a/packages/trilium-core/src/services/llm/host_providers.ts b/packages/trilium-core/src/services/llm/host_providers.ts
new file mode 100644
index 0000000000..452e7645a1
--- /dev/null
+++ b/packages/trilium-core/src/services/llm/host_providers.ts
@@ -0,0 +1,74 @@
+/**
+ * The host-provider registration seam, in a module of its own so hosts can
+ * register at startup without loading the LLM stack: `index.js` pulls every
+ * SDK-backed provider on import, while this file needs only a type. Keeping
+ * registration light is what lets the whole stack stay in lazy chunks until
+ * the first chat or MCP request (see the ESM `splitting` in build-utils).
+ */
+
+import type { LlmProvider } from "./types.js";
+
+/**
+ * Provider types core cannot build for itself, each mapped to the name the user
+ * knows it by.
+ *
+ * What they have in common is a dependency on the runtime around them rather
+ * than on an API key: both are CLIs this process spawns, authenticating through
+ * the host's own account plumbing rather than a key the user pastes in. Core
+ * runs in the browser too, where none of that exists, so the host that *can* do
+ * it registers a factory at startup with {@link registerHostProvider}. Where
+ * nothing registers one, selecting the provider fails with a message naming it
+ * rather than a bare "unknown type".
+ *
+ * Adding one means an entry here, a literal branch in `createProviderInstance`
+ * (see the note there on why it must be literal), and a registration in
+ * whichever hosts can serve it.
+ */
+export const HOST_PROVIDED_TYPES = {
+ /** Claude Pro/Max through the Claude Agent SDK, authenticated by `claude /login`. */
+ "claude-agent": "Claude Code",
+ /** GitHub Copilot through the Copilot CLI's ACP mode, authenticated by `copilot login`. */
+ "copilot-agent": "GitHub Copilot"
+} as const;
+
+/** A provider type from {@link HOST_PROVIDED_TYPES}. */
+export type HostProvidedType = keyof typeof HOST_PROVIDED_TYPES;
+
+/**
+ * A factory can resolve asynchronously so the host can dynamically import its
+ * provider module inside it — registration then costs nothing at startup.
+ */
+type HostProviderFactory = () => LlmProvider | Promise<LlmProvider>;
+
+const hostProviderFactories = new Map<string, HostProviderFactory>();
+
+/**
+ * Register the host's implementation of a provider core cannot construct.
+ * Called once at startup, before any chat can ask for it.
+ */
+export function registerHostProvider(type: HostProvidedType, factory: HostProviderFactory) {
+ hostProviderFactories.set(type, factory);
+}
+
+/**
+ * Forget every registered factory, putting the registry back to how a build that
+ * supplies none starts. For a runtime that re-initialises core (tests, an
+ * Electron reload), which registers again on the way back up.
+ */
+export function clearHostProviders() {
+ hostProviderFactories.clear();
+}
+
+/**
+ * Build a host-provided provider, or explain that this build has no one to build
+ * it. The type is always a literal from the call site — see
+ * `createProviderInstance` in `index.js` — never the string the user's config
+ * carried.
+ */
+export async function createHostProvider(type: HostProvidedType): Promise<LlmProvider> {
+ const factory = hostProviderFactories.get(type);
+ if (!factory) {
+ throw new Error(`The ${HOST_PROVIDED_TYPES[type]} provider is not available in this build.`);
+ }
+ return await factory();
+}
diff --git a/packages/trilium-core/src/services/llm/index.spec.ts b/packages/trilium-core/src/services/llm/index.spec.ts
index 772e40018d..3673bcf514 100644
--- a/packages/trilium-core/src/services/llm/index.spec.ts
+++ b/packages/trilium-core/src/services/llm/index.spec.ts
@@ -91,21 +91,21 @@ describe("llm/index provider registry", () => {
});
describe("getProvider", () => {
- it("returns the first provider when no id is given and caches it", () => {
+ it("returns the first provider when no id is given and caches it", async () => {
setProviders(TWO);
- const p1 = getProvider();
- const p2 = getProvider();
+ const p1 = await getProvider();
+ const p2 = await getProvider();
expect(p1).toBe(p2); // cached
expect((p1.constructor as any).lastArgs).toEqual(["k1", "https://proxy"]);
});
- it("returns the provider matching a given id", () => {
+ it("returns the provider matching a given id", async () => {
setProviders(TWO);
- const p = getProvider("o1");
+ const p = await getProvider("o1");
expect((p.constructor as any).lastArgs).toEqual(["k2", undefined]);
});
- it("instantiates each known provider type via its factory", () => {
+ it("instantiates each known provider type via its factory", async () => {
setProviders([
{ id: "a", name: "A", provider: "anthropic", apiKey: "ka" },
{ id: "o", name: "O", provider: "openai", apiKey: "ko" },
@@ -116,68 +116,69 @@ describe("llm/index provider registry", () => {
{ id: "lm", name: "LM", provider: "lmstudio", apiKey: "", baseURL: "http://box:1234/v1" },
{ id: "oc", name: "OC", provider: "openai-compatible", apiKey: "k", baseURL: "http://box:8080/v1" }
]);
- expect((getProvider("a").constructor as any).lastArgs).toEqual(["ka", undefined]);
- expect((getProvider("o").constructor as any).lastArgs).toEqual(["ko", undefined]);
- expect((getProvider("g").constructor as any).lastArgs).toEqual(["kg", undefined]);
+ const constructorArgs = async (id: string) => ((await getProvider(id)).constructor as any).lastArgs;
+ expect(await constructorArgs("a")).toEqual(["ka", undefined]);
+ expect(await constructorArgs("o")).toEqual(["ko", undefined]);
+ expect(await constructorArgs("g")).toEqual(["kg", undefined]);
// Its own class rather than the shared self-hosted one, despite speaking
// the same protocol — that is what gives its models a price.
- expect((getProvider("d").constructor as any).lastArgs).toEqual(["kd", undefined]);
+ expect(await constructorArgs("d")).toEqual(["kd", undefined]);
// The subscription provider takes no constructor args — auth is Claude Code's.
- expect((getProvider("c").constructor as any).lastArgs).toEqual([]);
+ expect(await constructorArgs("c")).toEqual([]);
// The three self-hosted cards share one class, which receives the card
// id so it knows which endpoint to probe and prefill.
- expect((getProvider("l").constructor as any).lastArgs).toEqual(["ollama", "", "http://ollama.lan:11434"]);
- expect((getProvider("lm").constructor as any).lastArgs).toEqual(["lmstudio", "", "http://box:1234/v1"]);
- expect((getProvider("oc").constructor as any).lastArgs).toEqual(["openai-compatible", "k", "http://box:8080/v1"]);
+ expect(await constructorArgs("l")).toEqual(["ollama", "", "http://ollama.lan:11434"]);
+ expect(await constructorArgs("lm")).toEqual(["lmstudio", "", "http://box:1234/v1"]);
+ expect(await constructorArgs("oc")).toEqual(["openai-compatible", "k", "http://box:8080/v1"]);
});
- it("throws when no providers are configured (null and empty array)", () => {
+ it("throws when no providers are configured (null and empty array)", async () => {
getOptionOrNullMock.mockReturnValue(null);
- expect(() => getProvider()).toThrow(/No LLM providers configured/);
+ await expect(getProvider()).rejects.toThrow(/No LLM providers configured/);
setProviders([]);
- expect(() => getProvider()).toThrow(/No LLM providers configured/);
+ await expect(getProvider()).rejects.toThrow(/No LLM providers configured/);
});
- it("throws when the requested id is not found", () => {
+ it("throws when the requested id is not found", async () => {
setProviders(TWO);
- expect(() => getProvider("nope")).toThrow(/not found: nope/);
+ await expect(getProvider("nope")).rejects.toThrow(/not found: nope/);
});
- it("throws for an unknown provider type", () => {
+ it("throws for an unknown provider type", async () => {
setProviders([{ id: "x", name: "X", provider: "mystery", apiKey: "k" }]);
- expect(() => getProvider("x")).toThrow(/Unknown LLM provider type: mystery/);
+ await expect(getProvider("x")).rejects.toThrow(/Unknown LLM provider type: mystery/);
});
- it("builds every host-provided type through the factory its host registered", () => {
+ it("builds every host-provided type through the factory its host registered", async () => {
// The catalog and the switch have to agree: a type listed in
// HOST_PROVIDED_TYPES with no branch of its own falls through to the
// default and reads as a typo, in a build that can in fact serve it.
const types = Object.keys(HOST_PROVIDED_TYPES);
setProviders(types.map((provider, index) => ({ id: `h${index}`, name: provider, provider, apiKey: "" })));
- types.forEach((type, index) => {
- expect(getProvider(`h${index}`)).toBeInstanceOf(hostProviderMocks[type]);
- });
+ for (const [index, type] of types.entries()) {
+ expect(await getProvider(`h${index}`)).toBeInstanceOf(hostProviderMocks[type]);
+ }
});
- it("names the provider when no host in this build can build it", () => {
+ it("names the provider when no host in this build can build it", async () => {
// What standalone hits for Claude Code: the type is real and the branch is
// there, but nothing registered a factory. Better than "unknown type",
// which would send the user looking for a typo.
clearHostProviders();
setProviders([{ id: "c", name: "C", provider: "claude-agent", apiKey: "" }]);
- expect(() => getProvider("c")).toThrow("The Claude Code provider is not available in this build.");
+ await expect(getProvider("c")).rejects.toThrow("The Claude Code provider is not available in this build.");
});
- it("drops cached instances when the llmProviders option changes", () => {
+ it("drops cached instances when the llmProviders option changes", async () => {
setProviders(TWO);
- const p1 = getProvider("a1");
+ const p1 = await getProvider("a1");
// Same config → still cached.
setProviders(TWO);
- expect(getProvider("a1")).toBe(p1);
+ expect(await getProvider("a1")).toBe(p1);
// Edited config (e.g. new API key) → cache self-invalidates.
setProviders([{ ...TWO[0], apiKey: "new-key" }, TWO[1]]);
- const p2 = getProvider("a1");
+ const p2 = await getProvider("a1");
expect(p2).not.toBe(p1);
expect((p2.constructor as any).lastArgs).toEqual(["new-key", "https://proxy"]);
});
@@ -202,15 +203,15 @@ describe("llm/index provider registry", () => {
});
describe("getProviderByType", () => {
- it("returns the first provider of the given type", () => {
+ it("returns the first provider of the given type", async () => {
setProviders(TWO);
- const p = getProviderByType("openai");
+ const p = await getProviderByType("openai");
expect((p.constructor as any).lastArgs).toEqual(["k2", undefined]);
});
- it("throws when no provider of that type is configured", () => {
+ it("throws when no provider of that type is configured", async () => {
setProviders(TWO);
- expect(() => getProviderByType("google")).toThrow(/No google provider configured/);
+ await expect(getProviderByType("google")).rejects.toThrow(/No google provider configured/);
});
});
diff --git a/packages/trilium-core/src/services/llm/index.ts b/packages/trilium-core/src/services/llm/index.ts
index 535ce8318c..fef3eb46ad 100644
--- a/packages/trilium-core/src/services/llm/index.ts
+++ b/packages/trilium-core/src/services/llm/index.ts
@@ -1,13 +1,11 @@
import { getLog } from "../../services/log.js";
import optionService from "../../services/options.js";
-import { AnthropicProvider } from "./providers/anthropic.js";
-import { DeepSeekProvider } from "./providers/deepseek.js";
-import { GoogleProvider } from "./providers/google.js";
-import { LocalProvider } from "./providers/local.js";
-import { OpenAiProvider } from "./providers/openai.js";
+import { createHostProvider } from "./host_providers.js";
import type { LlmProvider, ModelInfo } from "./types.js";
+export { clearHostProviders, HOST_PROVIDED_TYPES, type HostProvidedType, registerHostProvider } from "./host_providers.js";
+
/**
* Configuration for a single LLM provider instance.
* This matches the structure stored in the llmProviders option.
@@ -30,64 +28,6 @@ export interface LlmProviderSetup {
/** Provider type identifiers that can be instantiated, for error messages. */
const PROVIDER_TYPES = ["anthropic", "openai", "google", "deepseek", "claude-agent", "copilot-agent", "ollama", "lmstudio", "openai-compatible"];
-/**
- * Provider types core cannot build for itself, each mapped to the name the user
- * knows it by.
- *
- * What they have in common is a dependency on the runtime around them rather
- * than on an API key: both are CLIs this process spawns, authenticating through
- * the host's own account plumbing rather than a key the user pastes in. Core
- * runs in the browser too, where none of that exists, so the host that *can* do
- * it registers a factory at startup with {@link registerHostProvider}. Where
- * nothing registers one, selecting the provider fails with a message naming it
- * rather than a bare "unknown type".
- *
- * Adding one means an entry here, a literal branch in
- * {@link createProviderInstance} (see the note there on why it must be literal),
- * and a registration in whichever hosts can serve it.
- */
-export const HOST_PROVIDED_TYPES = {
- /** Claude Pro/Max through the Claude Agent SDK, authenticated by `claude /login`. */
- "claude-agent": "Claude Code",
- /** GitHub Copilot through the Copilot CLI's ACP mode, authenticated by `copilot login`. */
- "copilot-agent": "GitHub Copilot"
-} as const;
-
-/** A provider type from {@link HOST_PROVIDED_TYPES}. */
-export type HostProvidedType = keyof typeof HOST_PROVIDED_TYPES;
-
-const hostProviderFactories = new Map<string, () => LlmProvider>();
-
-/**
- * Register the host's implementation of a provider core cannot construct.
- * Called once at startup, before any chat can ask for it.
- */
-export function registerHostProvider(type: HostProvidedType, factory: () => LlmProvider) {
- hostProviderFactories.set(type, factory);
-}
-
-/**
- * Forget every registered factory, putting the registry back to how a build that
- * supplies none starts. For a runtime that re-initialises core (tests, an
- * Electron reload), which registers again on the way back up.
- */
-export function clearHostProviders() {
- hostProviderFactories.clear();
-}
-
-/**
- * Build a host-provided provider, or explain that this build has no one to build
- * it. The type is always a literal from the call site — see
- * {@link createProviderInstance} — never the string the user's config carried.
- */
-function createHostProvider(type: HostProvidedType): LlmProvider {
- const factory = hostProviderFactories.get(type);
- if (!factory) {
- throw new Error(`The ${HOST_PROVIDED_TYPES[type]} provider is not available in this build.`);
- }
- return factory();
-}
-
/**
* Instantiate a provider from its type identifier.
*
@@ -100,33 +40,38 @@ function createHostProvider(type: HostProvidedType): LlmProvider {
* The host-provided factories are looked up too, but only ever under a literal
* spelled out in their branch below — the untrusted string picks the branch and
* goes no further.
+ *
+ * Each branch imports its provider module on first use: the SDK behind it must
+ * stay out of the startup path (and out of the standalone worker's boot) for
+ * users who never chat, so the import specifiers are literals inside the
+ * branches rather than at module scope.
*/
-function createProviderInstance(provider: string, apiKey: string, baseURL?: string): LlmProvider {
+async function createProviderInstance(provider: string, apiKey: string, baseURL?: string): Promise<LlmProvider> {
switch (provider) {
case "anthropic":
- return new AnthropicProvider(apiKey, baseURL);
+ return new (await import("./providers/anthropic.js")).AnthropicProvider(apiKey, baseURL);
case "openai":
- return new OpenAiProvider(apiKey, baseURL);
+ return new (await import("./providers/openai.js")).OpenAiProvider(apiKey, baseURL);
case "google":
- return new GoogleProvider(apiKey, baseURL);
+ return new (await import("./providers/google.js")).GoogleProvider(apiKey, baseURL);
// OpenAI-compatible on the wire, but carded separately from the generic
// custom endpoint so its models resolve against the committed price table.
case "deepseek":
- return new DeepSeekProvider(apiKey, baseURL);
+ return new (await import("./providers/deepseek.js")).DeepSeekProvider(apiKey, baseURL);
// Subscription providers, whose credentials belong to the host rather than
// to a key the user pastes in — so the host builds them. One branch each,
// naming its own type: see HOST_PROVIDED_TYPES.
case "claude-agent":
- return createHostProvider("claude-agent");
+ return await createHostProvider("claude-agent");
case "copilot-agent":
- return createHostProvider("copilot-agent");
+ return await createHostProvider("copilot-agent");
// Self-hosted endpoints. The three cards differ only in the URL and setup
// hint the UI prefills; they all speak the OpenAI-compatible API, with
// Ollama and LM Studio additionally offering a richer native listing.
case "ollama":
case "lmstudio":
case "openai-compatible":
- return new LocalProvider(provider, apiKey, baseURL);
+ return new (await import("./providers/local.js")).LocalProvider(provider, apiKey, baseURL);
default:
throw new Error(`Unknown LLM provider type: ${provider}. Available: ${PROVIDER_TYPES.join(", ")}`);
}
@@ -166,7 +111,7 @@ function getConfiguredProviders(): LlmProviderSetup[] {
* Get a provider instance by its configuration ID.
* If no ID is provided, returns the first configured provider.
*/
-export function getProvider(providerId?: string): LlmProvider {
+export async function getProvider(providerId?: string): Promise<LlmProvider> {
const configs = getConfiguredProviders();
if (configs.length === 0) {
@@ -188,7 +133,7 @@ export function getProvider(providerId?: string): LlmProvider {
}
// Create new provider instance
- const provider = createProviderInstance(config.provider, config.apiKey, config.baseURL);
+ const provider = await createProviderInstance(config.provider, config.apiKey, config.baseURL);
cachedProviders[config.id] = provider;
return provider;
}
@@ -196,7 +141,7 @@ export function getProvider(providerId?: string): LlmProvider {
/**
* Get the first configured provider of a specific type (e.g., "anthropic").
*/
-export function getProviderByType(providerType: string): LlmProvider {
+export async function getProviderByType(providerType: string): Promise<LlmProvider> {
const configs = getConfiguredProviders();
const config = configs.find(c => c.provider === providerType);
@@ -224,7 +169,7 @@ export function hasConfiguredProviders(): boolean {
* dynamic listing.
*/
export async function listProviderModels(provider: string, apiKey: string, baseURL?: string): Promise<ModelInfo[]> {
- const instance = createProviderInstance(provider, apiKey, baseURL);
+ const instance = await createProviderInstance(provider, apiKey, baseURL);
const models = await (instance.listModels?.() ?? instance.getAvailableModels());
// Tag the default-selected set here so the recommendation rule lives on the
// server — in the provider that owns its model id shape — rather than in the
diff --git a/packages/trilium-core/src/services/llm/tools/index.ts b/packages/trilium-core/src/services/llm/tools/index.ts
index 15b56af2d3..765bf6d4f8 100644
--- a/packages/trilium-core/src/services/llm/tools/index.ts
+++ b/packages/trilium-core/src/services/llm/tools/index.ts
@@ -17,6 +17,7 @@ import { attributeTools } from "./attribute_tools.js";
import { hierarchyTools } from "./hierarchy_tools.js";
import { iconTools } from "./icon_tools.js";
import { noteTools } from "./note_tools.js";
+import { drainToolRegistryLoaders } from "./registration.js";
import { skillTools } from "./skill_tools.js";
import type { ToolRegistry } from "./tool_registry.js";
@@ -44,9 +45,25 @@ export const allToolRegistries: ToolRegistry[] = [
* or MCP tool listing, both of which read {@link allToolRegistries} eagerly.
* Registering the same registry twice is a no-op, so a host that initialises
* more than once (tests, Electron reloads) does not duplicate its tools.
+ *
+ * A host whose registry should stay out of the startup path registers a loader
+ * with `registerToolRegistryLoader` (see `registration.js`) instead of calling
+ * this directly.
*/
export function registerToolRegistry(registry: ToolRegistry) {
if (!allToolRegistries.includes(registry)) {
allToolRegistries.push(registry);
}
}
+
+/**
+ * Loads every host-registered lazy registry into {@link allToolRegistries}.
+ * The chat pipeline and the MCP server call this at the start of their async
+ * entry points, which is what lets `buildTools()` and other readers stay
+ * synchronous: by the time they iterate the array, it is complete.
+ */
+export async function resolveToolRegistries(): Promise<void> {
+ for (const loader of drainToolRegistryLoaders()) {
+ registerToolRegistry(await loader());
+ }
+}
diff --git a/packages/trilium-core/src/services/llm/tools/registration.ts b/packages/trilium-core/src/services/llm/tools/registration.ts
new file mode 100644
index 0000000000..d18d812249
--- /dev/null
+++ b/packages/trilium-core/src/services/llm/tools/registration.ts
@@ -0,0 +1,30 @@
+/**
+ * The lazy half of the tool-registry seam, in a module of its own so hosts can
+ * register at startup without loading the tool stack: `index.js` pulls every
+ * core registry (and through `tool_registry.js` the AI SDK and zod) on import,
+ * while this file needs only a type. A host registers a loader here; the first
+ * chat turn or MCP request resolves it via `resolveToolRegistries()` in
+ * `index.js`, and the registry's module loads then, not at startup.
+ */
+
+import type { ToolRegistry } from "./tool_registry.js";
+
+/** Imports and returns a host's tool registry on first use. */
+export type ToolRegistryLoader = () => Promise<ToolRegistry>;
+
+const pendingLoaders: ToolRegistryLoader[] = [];
+
+/**
+ * Register a loader for a host-provided tool registry. Called once at startup;
+ * registering again after a resolve (tests, an Electron reload) queues the
+ * loader again, and the duplicate collapses when it resolves to the same
+ * registry instance.
+ */
+export function registerToolRegistryLoader(loader: ToolRegistryLoader) {
+ pendingLoaders.push(loader);
+}
+
+/** Hands the queued loaders to `resolveToolRegistries()`, emptying the queue. */
+export function drainToolRegistryLoaders(): ToolRegistryLoader[] {
+ return pendingLoaders.splice(0, pendingLoaders.length);
+}