7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.
TypeScript
feat(router): task-type-aware routing (x-freellmapi-task-type header + bandit weight bias) (#1128)
suantea (asuan_suan)e29b415b79b7
diff --git a/docs/api/01-rest-api.md b/docs/api/01-rest-api.mdindex 947c0b5a..f1295b31 100644--- a/docs/api/01-rest-api.md+++ b/docs/api/01-rest-api.md@@ -288,6 +288,8 @@ The opt-in response cache can be toggled per request with `X-FreeLLM-Cache: on|oWhen [prompt compression](../compression/01-compression-pipeline.md) is enabled, `X-FreeLLM-Compress: off|on|lossless|standard|aggressive` can disable or lower the configured mode for one request. It cannot raise the operator's configured mode. The response reports the effective mode and estimated savings, for example `X-FreeLLM-Compress: standard; saved~=1840`.+`X-FreeLLM-Task-Type: code|chat|auto` tells the router what kind of turn this is, so it can bias the routing weights: `code` trades some of the speed weight for intelligence (a wrong answer costs more than a slow one), `chat` does the reverse. It is a soft preference only — capability gates, the failover chain and the model catalog are untouched, and the shift is 30% of one axis. The default is `auto`: tool-bearing requests and messages carrying unambiguous code markers (a fenced block, a declaration, a stack frame, a `file.ts:214` reference) are treated as `code`, and everything else keeps the preset weights exactly as configured. The `fastest`, `reliable` and `custom` routing strategies ignore the header — those are an explicit operator choice about where on the axis to sit.+## Embeddings`/v1/embeddings` is OpenAI-compatible, with one deliberate difference from chat routing: **failover never crosses models.** Vectors from different models live in incompatible spaces — silently switching models would corrupt any vector store built on top of the proxy. So embeddings route by **family** (one model identity + dimension), and failover only walks the providers serving that same family.diff --git a/docs/api/CHANGELOG.md b/docs/api/CHANGELOG.mdindex 124c35ad..dbb59e67 100644--- a/docs/api/CHANGELOG.md+++ b/docs/api/CHANGELOG.md@@ -2,6 +2,10 @@Doc revision history for `docs/api/`, seeded from API-relevant commits.+## 2026-09-03++- **docs(api): document the task-type routing header** — added `X-FreeLLM-Task-Type: code|chat|auto` (#1127) to the per-request headers section, with the weight shift it applies and the strategies that ignore it.+## 2026-08-25- **docs(api): document video platforms, free-tier budget, and backups APIs** — expanded `POST /v1/videos/generations` with the bounded pollinations/huggingface surface (5-minute timeout, fal queue); added `GET /api/free-tier` (pool-deduped monthly budget) and `GET /api/backups` (DUMP_FORMAT=1, sha256 key fingerprint) endpoint reference.diff --git a/server/src/__tests__/lib/task-type.test.ts b/server/src/__tests__/lib/task-type.test.tsnew file mode 100644index 00000000..0f074b21--- /dev/null+++ b/server/src/__tests__/lib/task-type.test.ts@@ -0,0 +1,115 @@+import { describe, it, expect } from 'vitest';+import type { Request } from 'express';+import { parseTaskTypeHeader, deriveTaskType, resolveTaskType, TASK_TYPE_HEADER } from '../../lib/task-type.js';++function reqWith(headerValue: string | undefined): Request {+ const headers: Record<string, string | string[] | undefined> = {};+ if (headerValue !== undefined) headers[TASK_TYPE_HEADER] = headerValue;+ return { headers } as unknown as Request;+}++describe('parseTaskTypeHeader', () => {+ it('reads an explicit code/chat declaration', () => {+ expect(parseTaskTypeHeader(reqWith('code'))).toBe('code');+ expect(parseTaskTypeHeader(reqWith('chat'))).toBe('chat');+ });++ it('treats the header as case-insensitive and trims whitespace', () => {+ expect(parseTaskTypeHeader(reqWith(' CODE '))).toBe('code');+ expect(parseTaskTypeHeader(reqWith('Chat'))).toBe('chat');+ });++ it('accepts an explicit auto and defaults to auto for anything else', () => {+ expect(parseTaskTypeHeader(reqWith('auto'))).toBe('auto');+ expect(parseTaskTypeHeader(reqWith('nonsense'))).toBe('auto');+ expect(parseTaskTypeHeader(reqWith(''))).toBe('auto');+ expect(parseTaskTypeHeader(reqWith(undefined))).toBe('auto');+ });++ it('takes the first value of a repeated header', () => {+ const req = { headers: { [TASK_TYPE_HEADER]: ['code', 'chat'] } } as unknown as Request;+ expect(parseTaskTypeHeader(req)).toBe('code');+ });+});++describe('deriveTaskType', () => {+ it('classifies a tool-bearing request as code', () => {+ expect(deriveTaskType([{ type: 'function' }], [{ role: 'user', content: 'hello' }])).toBe('code');+ });++ it('classifies a code-marked last user message as code', () => {+ const codeSamples = [+ '```python\nprint(1)\n```',+ 'const x = 1;',+ 'let total = items.length',+ 'why does function parse(input) return null here?',+ 'def build_index(rows):',+ 'class Cache extends Map {',+ "import { render } from 'react-dom'",+ 'from pathlib import Path',+ "const fs = require('fs')",+ '#include <stdio.h>',+ '$ npm run build fails on a clean checkout',+ 'it blows up at server/src/services/router.ts:214',+ 'crash in main.c:42',+ ' at handle (/app/dist/index.js:12:9)',+ 'Traceback (most recent call last)',+ ' File "app/main.py", line 30',+ ];+ for (const sample of codeSamples) {+ expect(deriveTaskType(undefined, [{ role: 'user', content: sample }]), sample).toBe('code');+ }+ });++ it('does not mistake ordinary prose for code (#1127)', () => {+ // The derivation is silent, so it has to err toward chat: these all matched+ // the first revision's bare-word markers (function/class/cargo/docker/=>).+ const proseSamples = [+ 'What class of animals are dolphins?',+ 'Explain the function of the mitochondria.',+ 'Summarize this article about cargo ships',+ 'Can you refactor my morning routine to fit a gym session?',+ 'Which method works best for brewing coffee at home?',+ 'I want to import olive oil from Italy, what paperwork do I need?',+ 'Write a short poem about a docker on the night shift',+ 'Is git a common surname in Norway?',+ 'The curl of a vector field, explained simply',+ 'She teaches a class on Renaissance painting',+ 'Chapter 4.c: what does the clause actually mean?',+ 'Rent is $ 1400 a month, is that reasonable?',+ ];+ for (const sample of proseSamples) {+ expect(deriveTaskType(undefined, [{ role: 'user', content: sample }]), sample).toBe('chat');+ }+ });++ it('falls back to chat for plain text, even with earlier code messages', () => {+ expect(deriveTaskType(undefined, [{ role: 'user', content: 'what is the weather in Berlin?' }])).toBe('chat');+ // Only the LAST user message counts — a stale code block from a previous turn is not intent.+ expect(deriveTaskType(undefined, [+ { role: 'user', content: '```js\nconst a = 1;\n```' },+ { role: 'assistant', content: 'done' },+ { role: 'user', content: 'thanks!' },+ ])).toBe('chat');+ });++ it('handles missing tools/messages and array content blocks', () => {+ expect(deriveTaskType(undefined, undefined)).toBe('chat');+ expect(deriveTaskType(undefined, [])).toBe('chat');+ expect(deriveTaskType(undefined, [{ role: 'user', content: [{ type: 'text', text: '```sql\nSELECT 1\n```' }] }])).toBe('code');+ });+});++describe('resolveTaskType', () => {+ it('explicit declaration always wins over derivation', () => {+ expect(resolveTaskType(reqWith('chat'), [{ type: 'function' }], [{ role: 'user', content: '```js\nx\n```' }])).toBe('chat');+ expect(resolveTaskType(reqWith('code'), undefined, [{ role: 'user', content: 'hi' }])).toBe('code');+ });++ it('auto derives code upward and stays undefined otherwise (preset weights untouched)', () => {+ expect(resolveTaskType(reqWith('auto'), undefined, [{ role: 'user', content: '```rust\nfn main() {}\n```' }])).toBe('code');+ expect(resolveTaskType(reqWith('auto'), [{ type: 'function' }], undefined)).toBe('code');+ expect(resolveTaskType(reqWith('auto'), undefined, [{ role: 'user', content: 'hello there' }])).toBeUndefined();+ expect(resolveTaskType(reqWith(undefined), undefined, [{ role: 'user', content: 'hello there' }])).toBeUndefined();+ });+});diff --git a/server/src/__tests__/routes/responses.test.ts b/server/src/__tests__/routes/responses.test.tsindex 58fc0157..9af29494 100644--- a/server/src/__tests__/routes/responses.test.ts+++ b/server/src/__tests__/routes/responses.test.ts@@ -83,6 +83,44 @@ describe('POST /v1/responses (#96)', () => {expect(mockRouteRequest.mock.calls.at(-1)?.[3]).toBe(true);});+ it('forwards a declared x-freellm-task-type to the router as the task arg (#1127)', async () => {+ mockRouteRequest.mockClear();+ mockRouteRequest.mockReturnValue(fakeRoute({+ async chatCompletion() {+ return {+ id: 'c', object: 'chat.completion', created: 0, model: 'fake-model',+ choices: [{ index: 0, message: { role: 'assistant', content: 'ok' }, finish_reason: 'stop' }],+ usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },+ };+ },+ async *streamChatCompletion() { /* unused */ },+ }));++ const { status } = await post(app, '/v1/responses', { input: 'refactor this function' }, key, { 'x-freellm-task-type': 'code' });+ expect(status).toBe(200);+ // routeRequest arg [10] is the task type.+ expect(mockRouteRequest.mock.calls.at(-1)?.[10]).toBe('code');+ });++ it('leaves the task arg undefined when no task header is sent (#1127)', async () => {+ mockRouteRequest.mockClear();+ mockRouteRequest.mockReturnValue(fakeRoute({+ async chatCompletion() {+ return {+ id: 'c', object: 'chat.completion', created: 0, model: 'fake-model',+ choices: [{ index: 0, message: { role: 'assistant', content: 'ok' }, finish_reason: 'stop' }],+ usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },+ };+ },+ async *streamChatCompletion() { /* unused */ },+ }));++ const { status } = await post(app, '/v1/responses', { input: 'hello there' }, key);+ expect(status).toBe(200);+ // routeRequest arg [10] is the task type; absent header ⇒ undefined ⇒ preset weights untouched.+ expect(mockRouteRequest.mock.calls.at(-1)?.[10]).toBeUndefined();+ });+it('rejects a file_id-only input_image with 422 before routing (no Files backend)', async () => {mockRouteRequest.mockClear();const { status, text } = await post(app, '/v1/responses', {diff --git a/server/src/__tests__/services/router-task-type.test.ts b/server/src/__tests__/services/router-task-type.test.tsnew file mode 100644index 00000000..b2d8be68--- /dev/null+++ b/server/src/__tests__/services/router-task-type.test.ts@@ -0,0 +1,157 @@+import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';+import {+ routeRequest, setRoutingStrategy, refreshStatsCache,+} from '../../services/router.js';+import { getDb, initDb } from '../../db/index.js';+import { addToActiveChain } from '../helpers/chain.js';+import * as ratelimit from '../../services/ratelimit.js';++vi.mock('../../services/ratelimit.js', async () => {+ const actual = await vi.importActual('../../services/ratelimit.js');+ return {+ ...actual,+ canMakeRequest: vi.fn(() => true),+ canUseTokens: vi.fn(() => true),+ isOnCooldown: vi.fn(() => false),+ };+});++vi.mock('../../lib/crypto.js', async () => {+ const actual = await vi.importActual('../../lib/crypto.js');+ return { ...actual, decrypt: vi.fn(() => 'mocked-api-key') };+});++const ORIGINAL_DEV_MODE = process.env.DEV_MODE;+const ORIGINAL_NODE_ENV = process.env.NODE_ENV;++// Same seed shape as router-bandit.test.ts, plus a configurable speed_rank so+// the two fixture models differ on BOTH axes (intelligence and speed).+function addModel(opts: {+ platform: string; modelId: string; name: string;+ intelligenceRank: number; speedRank: number; sizeLabel: string; budget: string; priority: number;+}): number {+ const db = getDb();+ db.prepare(`+ INSERT INTO models (platform, model_id, display_name, intelligence_rank, speed_rank, size_label, monthly_token_budget, enabled, supports_vision, supports_tools)+ VALUES (?, ?, ?, ?, ?, ?, ?, 1, 0, 1)+ `).run(opts.platform, opts.modelId, opts.name, opts.intelligenceRank, opts.speedRank, opts.sizeLabel, opts.budget);+ const id = (db.prepare('SELECT id FROM models WHERE platform = ? AND model_id = ?')+ .get(opts.platform, opts.modelId) as { id: number }).id;+ db.prepare('INSERT INTO fallback_config (model_db_id, priority, enabled) VALUES (?, ?, 1)').run(id, opts.priority);+ addToActiveChain(id, opts.priority);+ db.prepare(`+ INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)+ VALUES (?, 'k', 'enc', 'iv', 'tag', 'healthy', 1)+ `).run(opts.platform);+ return id;+}++function addHistory(platform: string, modelId: string, opts: {+ successes: number; failures: number; outTokens?: number; latencyMs?: number; ttfbMs?: number | null;+}) {+ const db = getDb();+ const ins = db.prepare(`+ INSERT INTO requests (platform, model_id, key_id, status, input_tokens, output_tokens, latency_ms, error, ttfb_ms)+ VALUES (?, ?, 1, ?, 0, ?, ?, ?, ?)+ `);+ for (let i = 0; i < opts.successes; i++) {+ ins.run(platform, modelId, 'success', opts.outTokens ?? 100, opts.latencyMs ?? 1000, null, opts.ttfbMs ?? null);+ }+ for (let i = 0; i < opts.failures; i++) {+ ins.run(platform, modelId, 'error', 0, opts.latencyMs ?? 1000, 'boom', opts.ttfbMs ?? null);+ }+}++function pickCounts(runs: number, task?: 'code' | 'chat'): Record<string, number> {+ const counts: Record<string, number> = {};+ for (let i = 0; i < runs; i++) {+ const r = routeRequest(100, undefined, undefined, false, false, undefined, undefined, false, undefined, 0, task);+ counts[r.modelId] = (counts[r.modelId] ?? 0) + 1;+ }+ return counts;+}++// The intelligence axis is min-max normalized to 1/0 across a two-model chain,+// so the code bias (0.325 intelligence) wins by ~0.15 score over the speed+// axis and the chat bias flips it the other way. Assert DIRECTION (not a fixed+// ratio) with enough runs that the ~55/45 split is well past noise.+const TASK_RUNS = 400;++// Fixture: A ("smart") is the smarter but much slower model, B ("fast") the+// weaker but blazing-fast one. Equal reliability (all successes) and equal+// budgets, so the only differing axes are intelligence (A > B — rank 1 is the+// catalog's best) and speed (B >> A). The speed gap is intentionally extreme:+// intelligenceScore is min-max normalized across the chain, so with two models+// it is always 1 vs 0, and the speed axis must out-weigh that for the chat+// bias to flip the pick.+function seedSmartVsFast() {+ addModel({ platform: 'google', modelId: 'smart', name: 'Smart', intelligenceRank: 1, speedRank: 9, sizeLabel: 'Large', budget: '~50M', priority: 1 });+ addModel({ platform: 'groq', modelId: 'fast', name: 'Fast', intelligenceRank: 2, speedRank: 1, sizeLabel: 'Large', budget: '~50M', priority: 2 });+ addHistory('google', 'smart', { successes: 100, failures: 0, outTokens: 1, latencyMs: 60000, ttfbMs: 4999 });+ addHistory('groq', 'fast', { successes: 100, failures: 0, outTokens: 10000, latencyMs: 100, ttfbMs: 10 });+}++describe('task-type-aware routing', () => {+ beforeEach(() => {+ process.env.DEV_MODE = 'true';+ process.env.NODE_ENV = 'test';+ initDb(':memory:');+ getDb().exec('DELETE FROM fallback_config; DELETE FROM api_keys; DELETE FROM models; DELETE FROM requests;');+ vi.clearAllMocks();+ (ratelimit.canMakeRequest as any).mockReturnValue(true);+ (ratelimit.canUseTokens as any).mockReturnValue(true);+ (ratelimit.isOnCooldown as any).mockReturnValue(false);+ });++ afterEach(() => {+ if (ORIGINAL_DEV_MODE === undefined) delete process.env.DEV_MODE; else process.env.DEV_MODE = ORIGINAL_DEV_MODE;+ if (ORIGINAL_NODE_ENV === undefined) delete process.env.NODE_ENV; else process.env.NODE_ENV = ORIGINAL_NODE_ENV;+ });++ it('is off by default: no task argument splits the two axes evenly', () => {+ seedSmartVsFast();+ setRoutingStrategy('balanced');+ refreshStatsCache(getDb(), true);+ const counts = pickCounts(TASK_RUNS);+ // Balanced weights (0.25/0.25) make A and B comparable; neither axis dominates.+ expect(counts['smart'] ?? 0).toBeGreaterThan(0);+ expect(counts['fast'] ?? 0).toBeGreaterThan(0);+ });++ it('code tasks bias toward the smarter (slower) model', () => {+ seedSmartVsFast();+ setRoutingStrategy('balanced');+ refreshStatsCache(getDb(), true);+ const counts = pickCounts(TASK_RUNS, 'code');+ expect(counts['smart'] ?? 0).toBeGreaterThan(counts['fast'] ?? 0);+ });++ it('chat tasks bias toward the faster (weaker) model', () => {+ seedSmartVsFast();+ setRoutingStrategy('balanced');+ refreshStatsCache(getDb(), true);+ const counts = pickCounts(TASK_RUNS, 'chat');+ expect(counts['fast'] ?? 0).toBeGreaterThan(counts['smart'] ?? 0);+ });++ // Same fixture, but the fast model is now visibly less reliable, which puts+ // `fastest`'s pick close enough to the edge that the code bias WOULD flip it+ // (0.55/0.10 speed/intelligence becomes 0.385/0.265). It must not: `fastest`,+ // `reliable` and `custom` are the operator's explicit choice of where on the+ // axis to sit, so a per-request header does not get to rewrite them (#1127).+ // The other two exempt strategies are covered exactly in scoring.test.ts.+ function seedSmartVsFlakyFast() {+ addModel({ platform: 'google', modelId: 'smart', name: 'Smart', intelligenceRank: 1, speedRank: 9, sizeLabel: 'Large', budget: '~50M', priority: 1 });+ addModel({ platform: 'groq', modelId: 'fast', name: 'Fast', intelligenceRank: 2, speedRank: 1, sizeLabel: 'Large', budget: '~50M', priority: 2 });+ addHistory('google', 'smart', { successes: 100, failures: 0, outTokens: 1, latencyMs: 60000, ttfbMs: 4999 });+ addHistory('groq', 'fast', { successes: 50, failures: 50, outTokens: 10000, latencyMs: 100, ttfbMs: 10 });+ }++ it('leaves the fastest preset alone on a code task', () => {+ seedSmartVsFlakyFast();+ setRoutingStrategy('fastest');+ refreshStatsCache(getDb(), true);+ const counts = pickCounts(TASK_RUNS, 'code');+ expect(counts['fast'] ?? 0).toBeGreaterThan(counts['smart'] ?? 0);+ });+});diff --git a/server/src/__tests__/services/scoring.test.ts b/server/src/__tests__/services/scoring.test.tsindex 652aabf8..b0c992e5 100644--- a/server/src/__tests__/services/scoring.test.ts+++ b/server/src/__tests__/services/scoring.test.ts@@ -5,6 +5,7 @@ import {expectedReliability, SPEED_PRIOR, HEADROOM_FLOOR,isPeakHours, peakAdjustedWeights, hourInTimezone, isValidPeakHour, isValidTimezone,isPeakExemptStrategy, DEFAULT_PEAK_HOURS, PEAK_SPEED_TO_RELIABILITY,+ taskAdjustedWeights, isTaskExemptStrategy, TASK_WEIGHT_SHARE,type PeakHoursConfig,} from '../../services/scoring.js';@@ -368,3 +369,49 @@ describe('scoring: time-of-day dynamic ranking (#760)', () => {expect(peakAdjustedWeights(base, 'balanced', bad, new Date('2026-08-18T20:00:00Z')).weights).toEqual(base);});});++describe('scoring: task-type weight bias (#1127)', () => {+ it('moves speed onto intelligence for a code turn, and back for a chat turn', () => {+ const base = BANDIT_PRESETS.balanced;+ const code = taskAdjustedWeights(base, 'code', 'balanced');+ expect(code.adjusted).toBe(true);+ expect(code.weights.intelligence).toBeCloseTo(base.intelligence + base.speed * TASK_WEIGHT_SHARE, 5);+ expect(code.weights.speed).toBeCloseTo(base.speed - base.speed * TASK_WEIGHT_SHARE, 5);+ expect(code.weights.reliability).toBe(base.reliability);+ expect(code.weights.reliability + code.weights.speed + code.weights.intelligence).toBeCloseTo(1, 5);++ const chat = taskAdjustedWeights(base, 'chat', 'balanced');+ expect(chat.adjusted).toBe(true);+ expect(chat.weights.speed).toBeCloseTo(base.speed + base.intelligence * TASK_WEIGHT_SHARE, 5);+ expect(chat.weights.intelligence).toBeCloseTo(base.intelligence - base.intelligence * TASK_WEIGHT_SHARE, 5);+ expect(chat.weights.reliability + chat.weights.speed + chat.weights.intelligence).toBeCloseTo(1, 5);+ });++ it('exempts fastest, reliable and custom', () => {+ // fastest/reliable are the ends of the axis the operator already picked;+ // custom is the vector they set by hand. A per-request header must not+ // rewrite any of the three.+ for (const strategy of ['fastest', 'reliable', 'custom'] as const) {+ expect(isTaskExemptStrategy(strategy)).toBe(true);+ const base = strategy === 'custom'+ ? { reliability: 0.2, speed: 0.5, intelligence: 0.3 }+ : BANDIT_PRESETS[strategy];+ for (const task of ['code', 'chat'] as const) {+ const out = taskAdjustedWeights(base, task, strategy);+ expect(out.weights).toEqual(base);+ expect(out.adjusted).toBe(false);+ }+ }+ for (const strategy of ['balanced', 'smartest'] as const) {+ expect(isTaskExemptStrategy(strategy)).toBe(false);+ expect(taskAdjustedWeights(BANDIT_PRESETS[strategy], 'code', strategy).adjusted).toBe(true);+ }+ });++ it('is a no-op when the axis it would move is already zero', () => {+ const noSpeed = { reliability: 0.5, speed: 0, intelligence: 0.5 };+ expect(taskAdjustedWeights(noSpeed, 'code', 'balanced')).toEqual({ weights: noSpeed, adjusted: false });+ const noIntel = { reliability: 0.5, speed: 0.5, intelligence: 0 };+ expect(taskAdjustedWeights(noIntel, 'chat', 'balanced')).toEqual({ weights: noIntel, adjusted: false });+ });+});diff --git a/server/src/lib/task-type.ts b/server/src/lib/task-type.tsnew file mode 100644index 00000000..85fa8c48--- /dev/null+++ b/server/src/lib/task-type.ts@@ -0,0 +1,104 @@+// Task-type-aware routing (#1127): lets a client declare what kind of work a+// request is (code vs chat) so the router can bias the bandit weights toward+// quality or speed. A coding turn is quality-sensitive — a weak model+// "completes" it with wrong code the user must manually verify; a chat turn is+// budget/speed-sensitive. The declared header is the strongest signal; when it+// is absent or `auto`, a cheap bounded rule derives the type from the request+// shape (tool-bearing requests → code; code markers in the last user message →+// code). The result is a SOFT preference: capability gates and the failover+// chain are untouched, and `undefined` means "leave the preset weights alone".++import type { Request } from 'express';+import type { ChatMessage } from '@freellmapi/shared/types.js';+import { contentToString } from './content.js';++export type TaskType = 'code' | 'chat' | 'auto';++export const TASK_TYPE_HEADER = 'x-freellm-task-type';++// Bounded scan so a huge user message never costs the router a full read.+const MAX_SCAN_CHARS = 4000;++/**+ * Markers that a turn is about code. Every one of these is SYNTAX, not+ * vocabulary: an earlier revision matched bare English words (`function`,+ * `class`, `cargo`, `docker`, `git`) and a bare `=>`, which classified+ * "What class of animals are dolphins?" and "Explain the function of the+ * mitochondria" as code. Auto-derivation is silent — the user never asked for+ * it — so it has to err toward `chat`: a missed code turn just routes the way+ * it always did, while a false positive quietly re-weights ordinary chat+ * traffic. Hence: a marker has to be something that essentially cannot occur+ * in prose.+ */+const CODE_MARKERS: readonly RegExp[] = [+ // Fenced code block (with or without a language tag).+ /```/,+ // Shell prompt at the start of a line, followed by a command-looking word.+ // The word gate keeps "$ 20 per seat" out.+ /^[ \t]*\$[ \t]+[a-z][\w.\/-]*(?=[ \t]|$)/m,+ // Declarations and definitions.+ /\b(?:const|let|var)\s+[A-Za-z_$][\w$]*\s*=/,+ /\bfunction\s*\*?\s*[A-Za-z_$][\w$]*\s*\(/,+ /\bdef\s+[A-Za-z_]\w*\s*\(/,+ // `class Foo:` / `class Foo {` / `class Foo(Base)` at the start of a line,+ // or `class Foo extends Bar` anywhere. Bare "class" never matches.+ /^[ \t]*class\s+[A-Za-z_$][\w$]*\s*[({:]/m,+ /\bclass\s+[A-Za-z_$][\w$]*\s+extends\s+[A-Za-z_$][\w$]*/,+ // ES imports: `import {` , or an import whose source is a quoted module.+ /\bimport\s*\{/,+ /\bimport\s[^\n;]{0,200}?\bfrom\s*['"][^'"\n]+['"]/,+ // Python: `from x.y import z` at the start of a line.+ /^[ \t]*from\s+[A-Za-z_.][\w.]*\s+import\s+\S/m,+ /\brequire\s*\(\s*['"]/,+ /^[ \t]*#\s*include\s*[<"]/m,+ // A source file path next to a line number: `src/router.ts:214`.+ /\b[\w.\/-]+\.(?:[jt]sx?|mjs|cjs|py|rb|go|rs|java|kt|swift|php|cs|cpp|cc|hpp|sh|sql|css|scss|html|vue|svelte|ya?ml|json|toml)\s*:\s*\d+/,+ // Same, for the one-letter C extensions — the base name must start with a+ // letter so an enumerated prose clause ("4.c: twelve") cannot match.+ /\b[A-Za-z_][\w.\/-]*\.[ch]\s*:\s*\d+/,+ // Stack frames: V8 (`at fn (file.js:1:2)`), Python (`File "x.py", line 3`).+ /^\s*at\s+\S.*\(.*:\d+:\d+\)/m,+ /^\s*File\s+"[^"\n]+",\s*line\s+\d+/m,+ /\bTraceback\s+\(most recent call last\)/,+];++/** Read the client-declared task type from the request header (default 'auto'). */+export function parseTaskTypeHeader(req: Request): TaskType {+ const raw = req.headers[TASK_TYPE_HEADER];+ const value = Array.isArray(raw) ? raw[0] : raw;+ const trimmed = value?.trim().toLowerCase();+ return trimmed === 'code' || trimmed === 'chat' || trimmed === 'auto' ? trimmed : 'auto';+}++/** True when `text` carries at least one unambiguous code marker. */+function hasCodeMarker(text: string): boolean {+ return CODE_MARKERS.some(re => re.test(text));+}++/** Derive a concrete task type from the request shape. Bounded, never throws. */+export function deriveTaskType(tools: unknown[] | undefined, messages: ChatMessage[] | undefined): 'code' | 'chat' {+ // Tool-bearing requests are overwhelmingly agent/coding turns.+ if (tools && tools.length > 0) return 'code';+ const lastUser = messages ? [...messages].reverse().find((m) => m.role === 'user') : undefined;+ const text = lastUser ? contentToString(lastUser.content) : '';+ if (text.length > 0 && hasCodeMarker(text.slice(0, MAX_SCAN_CHARS))) return 'code';+ return 'chat';+}++/**+ * Resolve the effective task bias for a request: the explicit header wins;+ * `auto` falls back to derivation. Returns undefined when there is no signal+ * worth biasing for — callers then leave the routing weights untouched, so the+ * default behaviour is unchanged. `auto` only ever biases UP toward `code`:+ * `chat` is the ordinary case, and silently re-weighting the bulk of traffic+ * in `auto` mode would defeat the point of an opt-in knob.+ */+export function resolveTaskType(+ req: Request,+ tools: unknown[] | undefined,+ messages: ChatMessage[] | undefined,+): 'code' | 'chat' | undefined {+ const declared = parseTaskTypeHeader(req);+ if (declared === 'code' || declared === 'chat') return declared;+ return deriveTaskType(tools, messages) === 'code' ? 'code' : undefined;+}diff --git a/server/src/routes/anthropic.ts b/server/src/routes/anthropic.tsindex c41a778f..33cbfbb6 100644--- a/server/src/routes/anthropic.ts+++ b/server/src/routes/anthropic.ts@@ -12,6 +12,7 @@ import type {import { routeRequest, resolveModelGroupCandidates, resolveStickyPreference, routingReserveTokens, type RouteResult, type ChainRow } from '../services/router.js';import { getSetting, getUnifiedApiKey } from '../db/index.js';import { contentToString } from '../lib/content.js';+import { resolveTaskType } from '../lib/task-type.js';import { repairToolArguments, toolSchemaMap } from '../lib/tool-args.js';import { invalidToolArgumentsError, invalidToolCallReasons, isToolArgumentValidationEnabled } from '../lib/tool-validate.js';import { rescueInlineToolCalls, startsWithDialectMarker, couldBecomeDialectMarker, containsDialectMarker } from '../lib/tool-call-rescue.js';@@ -613,7 +614,11 @@ anthropicRouter.post('/messages', async (req: Request, res: Response) => {attemptLog,clientGone: () => clientGone,abortInFlight: () => hedgeAbort.abort(newHedgeAbortError()),- route: () => routeRequest(estimatedTotal, state.skipKeys.size > 0 ? state.skipKeys : undefined, preferredModel, hasImage, wantsTools, state.skipModels.size > 0 ? state.skipModels : undefined, groupChain, false, state.skipPlatforms.size > 0 ? state.skipPlatforms : undefined, outputReserve),+ route: () => {+ // Task-type routing (#1127): same header/derivation as /chat/completions.+ const taskType = resolveTaskType(req, tools, messages);+ return routeRequest(estimatedTotal, state.skipKeys.size > 0 ? state.skipKeys : undefined, preferredModel, hasImage, wantsTools, state.skipModels.size > 0 ? state.skipModels : undefined, groupChain, false, state.skipPlatforms.size > 0 ? state.skipPlatforms : undefined, outputReserve, taskType);+ },dispatch: async (route, attempt, dispatchCtx) => {if (stream) {try {diff --git a/server/src/routes/proxy.ts b/server/src/routes/proxy.tsindex 7fb77b26..1c2ad4ef 100644--- a/server/src/routes/proxy.ts+++ b/server/src/routes/proxy.ts@@ -11,6 +11,7 @@ import multer from 'multer';import { getDb } from '../db/index.js';import { resolveAuth, prependSystemPrompt, type ResolvedAuth } from '../lib/system-prompt.js';import { contentToString, messageHasImage, normalizeOutboundContent, sanitizeResponse, truncateMessagesForGithub } from '../lib/content.js';+import { resolveTaskType } from '../lib/task-type.js';import { normalizeMessageImages } from '../lib/image-normalize.js';import { repairToolArguments, toolSchemaMap } from '../lib/tool-args.js';import { invalidToolArgumentsError, invalidToolCallReasons, isToolArgumentValidationEnabled } from '../lib/tool-validate.js';@@ -2005,7 +2006,11 @@ proxyRouter.post('/chat/completions', async (req: Request, res: Response) => {// model is on record). Turns where injection can't happen — every turn 1, and// sessions that never switched — pay no headroom tax.const routingEstimate = handoffPossible ? estimatedTotal + HANDOFF_MAX_TOKENS : estimatedTotal;- return routeRequest(routingEstimate, state.skipKeys.size > 0 ? state.skipKeys : undefined, preferredModel, hasImage, wantsTools, state.skipModels.size > 0 ? state.skipModels : undefined, groupChain ?? resolvedChain?.chain, samplingParams.response_format !== undefined, state.skipPlatforms.size > 0 ? state.skipPlatforms : undefined, outputReserve);+ // Task-type routing (#1127): the client can declare code/chat intent via+ // header; otherwise a bounded rule derives it (tools present / code+ // markers). undefined keeps the preset weights untouched.+ const taskType = resolveTaskType(req, tools, messages);+ return routeRequest(routingEstimate, state.skipKeys.size > 0 ? state.skipKeys : undefined, preferredModel, hasImage, wantsTools, state.skipModels.size > 0 ? state.skipModels : undefined, groupChain ?? resolvedChain?.chain, samplingParams.response_format !== undefined, state.skipPlatforms.size > 0 ? state.skipPlatforms : undefined, outputReserve, taskType);},dispatch: async (route, attempt, ctx) => {const modelKey = `${route.platform}:${route.modelId}`;diff --git a/server/src/routes/responses.ts b/server/src/routes/responses.tsindex 18c3f025..927947b7 100644--- a/server/src/routes/responses.ts+++ b/server/src/routes/responses.ts@@ -14,6 +14,7 @@ import { getDb } from '../db/index.js';import { resolveAuth, prependSystemPrompt } from '../lib/system-prompt.js';import { isUnifyEnabled, getModelGroups, resolveRequestedIdForDispatch } from '../services/model-groups.js';import { contentToString, messageHasImage } from '../lib/content.js';+import { resolveTaskType } from '../lib/task-type.js';import { normalizeMessageImages } from '../lib/image-normalize.js';import { repairToolArguments, toolSchemaMap } from '../lib/tool-args.js';import { invalidToolArgumentsError, invalidToolCallReasons, isToolArgumentValidationEnabled } from '../lib/tool-validate.js';@@ -771,7 +772,11 @@ responsesRouter.post('/responses', async (req: Request, res: Response) => {attemptLog,clientGone: () => clientGone,abortInFlight: () => hedgeAbort.abort(newHedgeAbortError()),- route: () => routeRequest(estimatedTotal, state.skipKeys.size > 0 ? state.skipKeys : undefined, preferredModel, hasImage, wantsTools, state.skipModels.size > 0 ? state.skipModels : undefined, groupChain, completionOpts.response_format !== undefined, state.skipPlatforms.size > 0 ? state.skipPlatforms : undefined, outputReserve),+ route: () => {+ // Task-type routing (#1127): same header/derivation as /chat/completions.+ const taskType = resolveTaskType(req, tools, messages);+ return routeRequest(estimatedTotal, state.skipKeys.size > 0 ? state.skipKeys : undefined, preferredModel, hasImage, wantsTools, state.skipModels.size > 0 ? state.skipModels : undefined, groupChain, completionOpts.response_format !== undefined, state.skipPlatforms.size > 0 ? state.skipPlatforms : undefined, outputReserve, taskType);+ },dispatch: async (route, attempt, ctx) => {traceRouteEvent('Responses', {event: attempt === 0 ? 'start' : 'next',diff --git a/server/src/services/router.ts b/server/src/services/router.tsindex 65743ce5..785b8cf7 100644--- a/server/src/services/router.ts+++ b/server/src/services/router.ts@@ -21,7 +21,7 @@ import {reliabilityPosterior, expectedReliability, sampleBeta,speedScore, intelligenceScore, intelligenceComposite, headroomFactor, rateWindowHeadroomFactor,rateLimitFactor, combineScore,- peakAdjustedWeights, isValidPeakHour, isValidTimezone,+ peakAdjustedWeights, taskAdjustedWeights, isValidPeakHour, isValidTimezone,DEFAULT_PEAK_HOURS, type PeakHoursConfig,observedSpeedRank, TIMEOUT_LATENCY_CAP_MS,type HeadroomThresholds,@@ -1022,12 +1022,12 @@ function scoreChainEntry(* faithful reflection of the user's picked strategy, not a re-sampled draw each* request. Priority mode is deterministic either way.*/-function orderChain(chain: ChainRow[], strategy: RoutingStrategy, sampled = true): ChainRow[] {+function orderChain(chain: ChainRow[], strategy: RoutingStrategy, sampled = true, task?: 'code' | 'chat'): ChainRow[] {// Tier first, always: it is the one ordering input that score must not be able// to override (see ChainRow.match_tier). Zero for every chain built anywhere// else, so this is a no-op outside slug-fallback resolution.const tier = (e: ChainRow) => e.match_tier ?? 0;- const weights = weightsFor(strategy);+ let weights = weightsFor(strategy);if (!weights) {// Legacy priority mode: manual chain order + the 429/failure penalty,// ascending.@@ -1064,6 +1064,16 @@ function orderChain(chain: ChainRow[], strategy: RoutingStrategy, sampled = true.map(x => x.e);}+ // Task-type bias (#1127): a client-declared/derived task type moves part of+ // one axis onto the other (code: speed → intelligence; chat: the reverse).+ // Applied AFTER the peak-hours adjustment, on the same weights the rest of+ // the chain scores with; opt-in, so absent a signal the preset stands.+ // `fastest`, `reliable` and `custom` are exempt (see TASK_EXEMPT_STRATEGIES).+ if (task) {+ const adjusted = taskAdjustedWeights(weights, task, strategy);+ weights = adjusted.adjusted ? adjusted.weights : weights;+ }+const composites = chain.map(e => intelligenceComposite(e.size_label, e.intelligence_rank));const intelMin = composites.length ? Math.min(...composites) : 0;const intelMax = composites.length ? Math.max(...composites) : 0;@@ -1878,7 +1888,7 @@ export function resolveFusionCandidate(modelId: string): FusionCandidate | nullreturn null;}-export function routeRequest(estimatedTokens = 1000, skipKeys?: Set<string>, preferredModelDbId?: number, requireVision = false, requireTools = false, skipModels?: Set<number>, prefetchedChain?: ChainRow[], requireStructured = false, skipPlatforms?: Set<string>, exactOutputReserve = 0): RouteResult {+export function routeRequest(estimatedTokens = 1000, skipKeys?: Set<string>, preferredModelDbId?: number, requireVision = false, requireTools = false, skipModels?: Set<number>, prefetchedChain?: ChainRow[], requireStructured = false, skipPlatforms?: Set<string>, exactOutputReserve = 0, task?: 'code' | 'chat'): RouteResult {const db = getDb();const strategy = getRoutingStrategy();@@ -1886,7 +1896,7 @@ export function routeRequest(estimatedTokens = 1000, skipKeys?: Set<string>, preconst chain = (prefetchedChain ?? getActiveChain(db)).filter(e => e.enabled);- const sortedChain = orderChain(chain, strategy);+ const sortedChain = orderChain(chain, strategy, true, task);// Exploration toggle (#685/#707 follow-up): when enabled, give a model with// no reliability/speed samples a guaranteed chance to be tried, so it stopsdiff --git a/server/src/services/scoring.ts b/server/src/services/scoring.tsindex 8b0324ed..8b905a45 100644--- a/server/src/services/scoring.ts+++ b/server/src/services/scoring.ts@@ -193,6 +193,62 @@ export function peakAdjustedWeights(};}+// ── Task-type weight bias (#1127) ─────────────────────────────────────────+// A coding turn is quality-sensitive, a chat turn is budget/speed-sensitive.+// When a request declares (or derives) a task type, move part of one axis onto+// the other — code: speed → intelligence, chat: intelligence → speed. The+// deltas are deliberately conservative and the whole thing is opt-in: no+// header (or `auto` with no code signal) leaves the preset weights untouched.+export const TASK_WEIGHT_SHARE = 0.3;++/**+ * Presets exempt from the task-type bias.+ *+ * The same reasoning as `PEAK_EXEMPT_STRATEGIES` (see above), plus `custom`:+ * `fastest` and `reliable` are the ends of the axis an operator picks when they+ * have already decided which end they want, and rewriting them turns one preset+ * into a noisy copy of another; `custom` is the operator's hand-set sliders, the+ * one weight vector they typed themselves, so a per-request header must not move+ * it. The bias therefore applies only to the mixed presets (`balanced`,+ * `smartest`). `priority` never reaches here — it has no weight vector at all.+ */+export const TASK_EXEMPT_STRATEGIES: readonly RoutingStrategy[] = [...PEAK_EXEMPT_STRATEGIES, 'custom'];++/** Whether this strategy opts out of the task-type bias (see above). */+export function isTaskExemptStrategy(strategy: RoutingStrategy): boolean {+ return TASK_EXEMPT_STRATEGIES.includes(strategy);+}++export function taskAdjustedWeights(+ base: RoutingWeights,+ task: 'code' | 'chat',+ strategy: RoutingStrategy,+): { weights: RoutingWeights; adjusted: boolean } {+ if (isTaskExemptStrategy(strategy)) return { weights: base, adjusted: false };+ if (task === 'code') {+ const shift = base.speed * TASK_WEIGHT_SHARE;+ if (shift <= 0) return { weights: base, adjusted: false };+ return {+ weights: {+ reliability: base.reliability,+ speed: base.speed - shift,+ intelligence: base.intelligence + shift,+ },+ adjusted: true,+ };+ }+ const shift = base.intelligence * TASK_WEIGHT_SHARE;+ if (shift <= 0) return { weights: base, adjusted: false };+ return {+ weights: {+ reliability: base.reliability,+ speed: base.speed + shift,+ intelligence: base.intelligence - shift,+ },+ adjusted: true,+ };+}+// ── Reliability ───────────────────────────────────────────────────────────// Beta(1,1) prior = uniform: an unseen model is genuinely uncertain, not assumed// good or bad. With decay-weighted pseudo-counts the alpha/beta are continuous.