Skip to content

Turbocharge Claude Code, Cursor, Codex, Gemini & every coding agent: faster, cheaper, with contextual understanding specific to your codebase.

feat: add OrcaRouter as a first-class LLM provider (#267)

lovejones2914-spec4a0e457a903a
diff --git a/.env.example b/.env.example
index 2dafebf4..6c8a966a 100644
--- a/.env.example
+++ b/.env.example
@@ -9,7 +9,8 @@
# Bring your own key from whichever provider you use.
# --- Provider (required for `build --deep`) -----------------------------------
-# Wire format / SDK: "openai" (any OpenAI-compatible endpoint) or "anthropic".
+# Wire format / SDK: "openai" (any OpenAI-compatible endpoint), "anthropic",
+# "litellm" (a LiteLLM proxy), or "orcarouter" (the OrcaRouter AI gateway).
# GRAFT_PROVIDER=openai
# Your provider API key.
# GRAFT_API_KEY=
@@ -28,12 +29,23 @@
# GRAFT_PROVIDER=anthropic
# GRAFT_API_KEY=sk-ant-...
# GRAFT_MODEL=claude-sonnet-5
+#
+# OrcaRouter AI gateway (OpenAI-compatible, no base URL needed):
+# GRAFT_PROVIDER=orcarouter
+# GRAFT_API_KEY=sk-orca-...
+# GRAFT_MODEL=openai/gpt-4o-mini
# --- Deprecated (still honored as a fallback) ---------------------------------
# If GRAFT_API_KEY is unset, these legacy OpenRouter vars are used instead.
# OPENROUTER_API_KEY=sk-or-...
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# GRAFT_OPENROUTER_MODEL=openai/gpt-4o-mini
+#
+# If GRAFT_API_KEY and the OpenRouter fallback are both unset, OrcaRouter's
+# vars are honored as a second fallback.
+# ORCAROUTER_API_KEY=sk-orca-...
+# ORCAROUTER_BASE_URL=https://api.orcarouter.ai/v1
+# ORCAROUTER_MODEL=openai/gpt-4o-mini
# --- Graph location -----------------------------------------------------------
# Where the graph lives (default: <repo>/.context)
diff --git a/README.md b/README.md
index b5c4ecef..f1a28f40 100644
--- a/README.md
+++ b/README.md
@@ -107,7 +107,7 @@ Graft builds that understanding **once** and writes it into your repo as a folde
- **A real graph you can read.** No embeddings, no similarity search, no index to keep warm. The graph is a set of linked files your agent opens, greps, and follows, exactly the way it reads any other file in the repo.
- **A local cache, not a committed artifact.** `graft build` writes `graft/` and adds it to `.gitignore` — it's a regenerable local cache, like `node_modules`. What you commit is the small wiring `graft init` drops in (`.claude/`, `AGENTS.md`, the MCP config); each teammate runs `graft build` to generate their own graph. No database, no server, no setup.
- **Always fresh, automatically.** Every query rebuilds the graph against the working tree first — structural, `$0`, ~3ms when nothing moved — so `ask`/`grep`/`callers`/`skeleton`/`map` describe the code as it is right now, including uncommitted edits. `graft check` is a local freshness signal; there's no stale index to babysit.
-- **Your provider, your key, your model.** Summaries are written by any provider you choose — OpenAI, Anthropic (native), OpenRouter, Fireworks, Groq, a LiteLLM proxy, or a local model — under your own key. The structural code graph (`graft build`, `graft check`) is deterministic tree-sitter and never calls a model at all.
+- **Your provider, your key, your model.** Summaries are written by any provider you choose — OpenAI, Anthropic (native), OpenRouter, Fireworks, Groq, OrcaRouter, a LiteLLM proxy, or a local model — under your own key. The structural code graph (`graft build`, `graft check`) is deterministic tree-sitter and never calls a model at all.
<p align="center">
<picture>
@@ -240,7 +240,7 @@ _Summary, sources, links, and notes ship today in markdown nodes. The crux ships
## What runs where
- **On your machine, no key, no network:** the structural code graph. `graft build` (wiring graph + per-file cards), `graft check`, and `graft ask` are deterministic tree-sitter — they never call a model.
-- **Through your provider key:** the LLM-written parts — `graft build --deep` adds the concept nodes (file summaries + node synthesis) and the per-symbol summaries and cruxes. graft is vendor-neutral: set `GRAFT_PROVIDER` (`openai` for any OpenAI-compatible endpoint, or `anthropic` for the native API), your `GRAFT_API_KEY`, `GRAFT_MODEL`, and — for the `openai` wire format — `GRAFT_BASE_URL` to point at OpenRouter, Fireworks, Groq, a LiteLLM proxy, a local server, or OpenAI itself. Or pass `--provider/--model/--api-key/--base-url` on the command line. (`OPENROUTER_API_KEY` still works as a deprecated fallback.)
+- **Through your provider key:** the LLM-written parts — `graft build --deep` adds the concept nodes (file summaries + node synthesis) and the per-symbol summaries and cruxes. graft is vendor-neutral: set `GRAFT_PROVIDER` (`openai` for any OpenAI-compatible endpoint, `anthropic` for the native API, or `litellm` / `orcarouter` for a gateway that speaks the OpenAI-compatible format), your `GRAFT_API_KEY`, `GRAFT_MODEL`, and — for the `openai` wire format — `GRAFT_BASE_URL` to point at OpenRouter, Fireworks, Groq, a LiteLLM proxy, a local server, or OpenAI itself. Or pass `--provider/--model/--api-key/--base-url` on the command line. (`OPENROUTER_API_KEY` still works as a deprecated fallback, and `ORCAROUTER_API_KEY` as a second one.)
- **Anonymous usage stats** — the only network calls are the LLM requests you configured, a daily npm version check, and one batched usage ping. The ping carries buckets and fixed labels only: never your code, file paths, repo name, symbols, queries, or error messages. [`TELEMETRY.md`](TELEMETRY.md) is the complete list and `graft telemetry debug` prints exactly what your machine would send. Turn it off with `graft telemetry disable`, `DO_NOT_TRACK=1`, or by unchecking the box in `graft init`; it is off in CI and in any build from source.
See [`.env.example`](.env.example) for the full list of settings (model, base URL, graph directory).
diff --git a/src/ai/llm/factory.ts b/src/ai/llm/factory.ts
index 1d1a53ff..eb0adb44 100644
--- a/src/ai/llm/factory.ts
+++ b/src/ai/llm/factory.ts
@@ -8,13 +8,19 @@
* `litellm` is a convenience over `openai`: same wire format, but pointed at a
* LiteLLM proxy by default and paired with `/v1/models` auto-discovery
* (see litellm.ts), so one endpoint reaches 100+ providers.
+ *
+ * `orcarouter` is the same kind of convenience over `openai`, pointed at the
+ * OrcaRouter AI gateway by default (see orcarouter.ts) so its users get the
+ * gateway's routing, failover, and guardrails behind a named provider instead
+ * of a bare custom base URL.
*/
import type { ChatModel } from "./types.js";
import { OpenAIChatModel } from "./openai.js";
import { AnthropicChatModel } from "./anthropic.js";
import { LiteLLMChatModel } from "./litellm.js";
+import { OrcaRouterChatModel } from "./orcarouter.js";
-export type ProviderKind = "openai" | "anthropic" | "litellm";
+export type ProviderKind = "openai" | "anthropic" | "litellm" | "orcarouter";
export interface ChatModelConfig {
provider: ProviderKind;
@@ -43,6 +49,13 @@ export function createChatModel(cfg: ChatModelConfig): ChatModel {
baseUrl: cfg.baseUrl,
headers: cfg.headers,
});
+ case "orcarouter":
+ return new OrcaRouterChatModel({
+ apiKey: cfg.apiKey,
+ model: cfg.model,
+ baseUrl: cfg.baseUrl,
+ headers: cfg.headers,
+ });
default: {
const _exhaustive: never = cfg.provider;
throw new Error(`unknown provider: ${String(_exhaustive)}`);
diff --git a/src/ai/llm/orcarouter.ts b/src/ai/llm/orcarouter.ts
new file mode 100644
index 00000000..01b3a9d9
--- /dev/null
+++ b/src/ai/llm/orcarouter.ts
@@ -0,0 +1,54 @@
+/**
+ * OrcaRouter transport. The OrcaRouter gateway speaks the OpenAI-compatible
+ * wire format, so this reuses the OpenAI adapter's request/response translation
+ * and simply points it at the gateway. The value over a bare `openai` provider +
+ * `baseUrl` is a first-class, self-describing provider: a sensible default
+ * gateway address and `/v1/models` auto-discovery (see {@link listOrcaRouterModels}),
+ * so callers can enumerate whatever upstream models the gateway routes to —
+ * reaching 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, and more
+ * through one endpoint without a per-vendor code path.
+ */
+import OpenAI from "openai";
+import { OpenAIChatModel, type OpenAIChatModelOptions } from "./openai.js";
+import { transportRetries } from "./types.js";
+
+/** Public endpoint of the OrcaRouter AI gateway. */
+export const DEFAULT_ORCAROUTER_BASE_URL = "https://api.orcarouter.ai/v1";
+
+export type OrcaRouterChatModelOptions = OpenAIChatModelOptions;
+
+/**
+ * ChatModel backed by the OrcaRouter gateway. Inherits every translation detail
+ * from {@link OpenAIChatModel} (the gateway is OpenAI-compatible) and only
+ * changes the defaults: the gateway base URL and an `orcarouter:<model>` label.
+ */
+export class OrcaRouterChatModel extends OpenAIChatModel {
+ constructor(opts: OrcaRouterChatModelOptions) {
+ super({
+ ...opts,
+ baseUrl: opts.baseUrl ?? DEFAULT_ORCAROUTER_BASE_URL,
+ label: opts.label ?? `orcarouter:${opts.model}`,
+ });
+ }
+}
+
+/**
+ * Auto-discover the models the OrcaRouter gateway can route to via `GET
+ * /v1/models`. Returns the model ids so a caller can present/validate them
+ * instead of hardcoding a model string. A stub `client` may be injected for tests.
+ */
+export async function listOrcaRouterModels(opts: {
+ apiKey: string;
+ baseUrl?: string;
+ client?: OpenAI;
+}): Promise<string[]> {
+ const client =
+ opts.client ??
+ new OpenAI({
+ apiKey: opts.apiKey,
+ baseURL: opts.baseUrl ?? DEFAULT_ORCAROUTER_BASE_URL,
+ maxRetries: transportRetries(),
+ });
+ const res = await client.models.list();
+ return res.data.map((m) => m.id);
+}
diff --git a/src/ai/providers.ts b/src/ai/providers.ts
index 2f743713..c248a68d 100644
--- a/src/ai/providers.ts
+++ b/src/ai/providers.ts
@@ -55,6 +55,7 @@ export interface ResolvedConfig {
}
const OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1";
+const ORCAROUTER_BASE_URL = "https://api.orcarouter.ai/v1";
/** Per-provider default model. */
export const DEFAULT_MODELS: Record<ProviderKind, string> = {
@@ -62,6 +63,8 @@ export const DEFAULT_MODELS: Record<ProviderKind, string> = {
anthropic: "claude-sonnet-5",
// Provider-prefixed so the LiteLLM proxy routes it; override with GRAFT_MODEL.
litellm: "openai/gpt-4o-mini",
+ // Provider-prefixed so the OrcaRouter gateway routes it; override with GRAFT_MODEL.
+ orcarouter: "openai/gpt-4o-mini",
};
export const DEFAULTS = {
@@ -76,16 +79,22 @@ export function resolveConfig(config: EngineConfig = {}): ResolvedConfig {
const explicitKey = config.apiKey ?? env.GRAFT_API_KEY;
const legacyKey = env.OPENROUTER_API_KEY;
- const apiKey = explicitKey ?? legacyKey;
+ const apiKey = explicitKey ?? legacyKey ?? env.ORCAROUTER_API_KEY;
const usedLegacyEnv = !explicitKey && !!legacyKey;
const model =
- config.model ?? env.GRAFT_MODEL ?? env.GRAFT_OPENROUTER_MODEL ?? DEFAULT_MODELS[provider];
+ config.model ??
+ env.GRAFT_MODEL ??
+ env.GRAFT_OPENROUTER_MODEL ??
+ env.ORCAROUTER_MODEL ??
+ DEFAULT_MODELS[provider];
- let baseUrl = config.baseUrl ?? env.GRAFT_BASE_URL ?? env.OPENROUTER_BASE_URL;
+ let baseUrl = config.baseUrl ?? env.GRAFT_BASE_URL ?? env.OPENROUTER_BASE_URL ?? env.ORCAROUTER_BASE_URL;
// Back-compat: an existing setup with only OPENROUTER_API_KEY keeps hitting
// OpenRouter without any config change.
if (!baseUrl && provider === "openai" && usedLegacyEnv) baseUrl = OPENROUTER_BASE_URL;
+ // The orcarouter provider points at the gateway unless a base URL is given.
+ if (!baseUrl && provider === "orcarouter") baseUrl = ORCAROUTER_BASE_URL;
const headers =
provider === "openai" && baseUrl?.includes("openrouter.ai")
diff --git a/src/cli.ts b/src/cli.ts
index e7d1e694..c83afea6 100644
--- a/src/cli.ts
+++ b/src/cli.ts
@@ -89,7 +89,7 @@ program
.description("Build a repo's context graph as linked markdown, and keep it in sync with the code.")
.version(currentVersion, "-v, --version")
.option("--dir <path>", "context graph directory (default: <repo>/graft)")
- .option("--provider <name>", "LLM wire format: openai | anthropic (env GRAFT_PROVIDER)")
+ .option("--provider <name>", "LLM wire format: openai | anthropic | litellm | orcarouter (env GRAFT_PROVIDER)")
.option("--model <id>", "model id for the LLM pass (env GRAFT_MODEL)")
.option("--api-key <key>", "provider API key (env GRAFT_API_KEY)")
.option("--base-url <url>", "OpenAI-compatible endpoint URL (env GRAFT_BASE_URL)");
diff --git a/src/index.ts b/src/index.ts
index 6f491457..de5ed4d3 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -34,6 +34,8 @@ export type { ChatModel, ChatRequest, ChatResponse, Message, ToolSpec, ToolCall,
export { createChatModel, type ProviderKind, type ChatModelConfig } from "./ai/llm/factory.js";
export { OpenAIChatModel } from "./ai/llm/openai.js";
export { AnthropicChatModel } from "./ai/llm/anthropic.js";
+export { LiteLLMChatModel, listLiteLLMModels, DEFAULT_LITELLM_BASE_URL } from "./ai/llm/litellm.js";
+export { OrcaRouterChatModel, listOrcaRouterModels, DEFAULT_ORCAROUTER_BASE_URL } from "./ai/llm/orcarouter.js";
// Engine ops, for advanced/custom setups.
export { ChatSynthesizer } from "./ai/synthesize.js";
diff --git a/test/orcarouter-adapter.test.ts b/test/orcarouter-adapter.test.ts
new file mode 100644
index 00000000..445dd7fd
--- /dev/null
+++ b/test/orcarouter-adapter.test.ts
@@ -0,0 +1,65 @@
+/**
+ * Network-free tests for the OrcaRouter provider. The OrcaRouter gateway is
+ * OpenAI-compatible, so the adapter reuses the OpenAI translation (verified in
+ * llm-adapters.test.ts); here we assert the orcarouter-specific behavior: reuse
+ * of that translation, the `orcarouter:` label, factory wiring, the default
+ * base URL, and `/v1/models` auto-discovery. Stub clients mean no key and no
+ * network.
+ */
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import type OpenAI from "openai";
+import {
+ OrcaRouterChatModel,
+ listOrcaRouterModels,
+ DEFAULT_ORCAROUTER_BASE_URL,
+} from "../src/ai/llm/orcarouter.js";
+import { createChatModel } from "../src/ai/llm/factory.js";
+
+function fakeOpenAI(resp: unknown) {
+ const box: { params?: any } = {};
+ const client = {
+ chat: { completions: { create: async (params: any) => ((box.params = params), resp) } },
+ } as unknown as OpenAI;
+ return { client, box };
+}
+
+function openAiResp(over: Partial<any> = {}): any {
+ return {
+ choices: [{ message: { content: "ok", tool_calls: [] }, finish_reason: "stop" }],
+ usage: { prompt_tokens: 10, completion_tokens: 5 },
+ ...over,
+ };
+}
+
+test("orcarouter: reuses OpenAI-compatible translation and labels as orcarouter", async () => {
+ const { client, box } = fakeOpenAI(openAiResp());
+ const m = new OrcaRouterChatModel({ apiKey: "x", model: "openai/gpt-4o-mini", client });
+ const res = await m.create({ messages: [{ role: "user", content: "hi" }], temperature: 0 });
+ assert.equal(box.params.model, "openai/gpt-4o-mini");
+ assert.equal(box.params.temperature, 0); // OpenAI-compatible: temperature forwarded
+ assert.equal(res.text, "ok");
+ assert.equal(m.label, "orcarouter:openai/gpt-4o-mini");
+});
+
+test("orcarouter: /v1/models auto-discovery returns the gateway's model ids", async () => {
+ const client = {
+ models: {
+ list: async () => ({
+ data: [{ id: "openai/gpt-4o-mini" }, { id: "anthropic/claude-sonnet-5" }],
+ }),
+ },
+ } as unknown as OpenAI;
+ const ids = await listOrcaRouterModels({ apiKey: "x", client });
+ assert.deepEqual(ids, ["openai/gpt-4o-mini", "anthropic/claude-sonnet-5"]);
+});
+
+test("orcarouter: factory builds an OrcaRouterChatModel for provider 'orcarouter'", () => {
+ const m = createChatModel({ provider: "orcarouter", apiKey: "x", model: "gpt-4o-mini" });
+ assert.ok(m instanceof OrcaRouterChatModel);
+ assert.equal(m.label, "orcarouter:gpt-4o-mini");
+});
+
+test("orcarouter: exposes a default gateway base URL", () => {
+ assert.equal(DEFAULT_ORCAROUTER_BASE_URL, "https://api.orcarouter.ai/v1");
+});