Skip to content

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

Make GPT-5.6 Luna the Auto default

Louis Beaumont5873eccd920b
diff --git a/packages/ai-gateway/src/handlers/chat.ts b/packages/ai-gateway/src/handlers/chat.ts
index d8ba3b9e7..897c455ec 100644
--- a/packages/ai-gateway/src/handlers/chat.ts
+++ b/packages/ai-gateway/src/handlers/chat.ts
@@ -10,17 +10,14 @@ import { isFlexEligible } from '../utils/latency';
import { routeTier, routerArm, TIER_HEAD } from './difficulty-router';
import { captureException } from '@sentry/cloudflare';
-// Auto model waterfall (INTERACTIVE) — leads with glm-5. Interactive is
-// latency-bound (a user is waiting), and on these models intelligence == reasoning
-// effort == latency: gemini-3.5-flash / gpt-5.x are only "smarter" at high effort,
-// which is slow (3-10s+); at chat speed they drop BELOW glm-5's IQ. glm-5 is the
-// fast/smart sweet spot for chat (~1.2s, AA 50, free Vertex MaaS). Smart reasoning
-// models live on AUTO_WATERFALL_BACKGROUND where latency doesn't matter.
+// Auto model waterfall (INTERACTIVE) — Luna leads while the OpenAI credit
+// allocation is available. Keep fast/free models behind it for resilience if
+// OpenAI is unavailable or rate-limited.
// Exported so tests can pin that every chain entry has a MODEL_PRICING match
// (otherwise served-model cost rows fall into the unknown-model estimate).
export const AUTO_WATERFALL = [
- 'glm-5', // fast (~1.2s) + AA 50, free Vertex MaaS — best fast/smart for latency-bound chat
- 'gpt-5.6-luna', // paid high-volume fallback — stronger when the fast free primary is unavailable
+ 'gpt-5.6-luna', // OpenAI-credit-funded default for Auto chat
+ 'glm-5', // fast/free Vertex MaaS fallback
'kimi-k2.5',
'glm-4.7',
'gemini-3-flash',
@@ -28,29 +25,22 @@ export const AUTO_WATERFALL = [
// Vision-capable models for requests containing images
export const AUTO_WATERFALL_VISION = [
- 'gemini-3.5-flash', // multimodal, leads on agent/vision benchmarks
- 'gpt-5.6-luna', // OpenAI vision fallback before the lower-capability free options
+ 'gpt-5.6-luna', // OpenAI-credit-funded default for Auto vision chat
+ 'gemini-3.5-flash', // multimodal fallback
'llama-4-maverick', // free (Vertex MaaS), 400B MoE, strong vision + reasoning
'gemini-3-flash', // near-free, good vision
'llama-4-scout', // free (Vertex MaaS), 109B MoE, decent vision fallback
'gemini-2.5-flash', // backup vision option
];
-// Background waterfall — for latency-tolerant traffic (pipes, daily summary,
-// suggestions) where no user is waiting. Leads with gemini-3.5-flash on the FLEX
-// tier: background is ~84% cache-reads, so flex's 0.1x cache discount makes it the
-// cheapest decent-quality option ($/Mtok ≈ glm-5 but with the cache discount glm-5
-// lacks). Measured (6/19): gpt-5.4 here cost $590/day vs $177 on flex for the SAME
-// traffic — 3.3x more, on the small OpenAI pool, for a marginal IQ gain (54 vs ~50-55)
-// that pipes don't need; reverted (#4285→). glm-5 + gemini-3-flash are standard-tier
-// fallbacks for when flex is throttled. Luna is a paid, high-volume fallback
-// immediately after the flex primary: it improves resilience without replacing
-// the proven cache-efficient default for every scheduled run. Pi's streaming
-// tool policy is forwarded by the OpenAI adapter, so this lane can use Luna for
-// tool-enabled pipes as intended.
+// Background waterfall — for pipes, summaries, and suggestions. Luna leads
+// while OpenAI credits are available; Gemini FLEX and Vertex MaaS remain
+// fallbacks for outages, quota limits, or later cost-control changes. Pi's
+// streaming tool policy is forwarded by the OpenAI adapter, so Luna supports
+// tool-enabled pipes on this lane.
export const AUTO_WATERFALL_BACKGROUND = [
- 'gemini-3.5-flash', // flex tier — cheapest on the cache-heavy background mix
- 'gpt-5.6-luna', // paid fallback for complex / tool-enabled pipe work
+ 'gpt-5.6-luna', // OpenAI-credit-funded default for Auto pipes
+ 'gemini-3.5-flash', // flex-tier fallback
'glm-5', // free Vertex MaaS fallback, standard tier
'gemini-3-flash', // near-free safety net
];
diff --git a/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts b/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts
index 95c68ef68..88550d00e 100644
--- a/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts
+++ b/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts
@@ -73,10 +73,10 @@ describe('routing chains stay priceable (no silent $0.01 fallback rows)', () =>
expect(getModelCost('gemini-3-flash', 1_000_000, 0)).toBeCloseTo(0.10, 10);
});
- it('places Luna behind the current auto primaries for text, vision, and pipes', () => {
+ it('uses Luna as the current Auto primary for text, vision, and pipes', () => {
const luna = 'gpt-5.6-luna';
- expect(AUTO_WATERFALL.indexOf('glm-5')).toBeLessThan(AUTO_WATERFALL.indexOf(luna));
- expect(AUTO_WATERFALL_VISION.indexOf('gemini-3.5-flash')).toBeLessThan(AUTO_WATERFALL_VISION.indexOf(luna));
- expect(AUTO_WATERFALL_BACKGROUND.indexOf('gemini-3.5-flash')).toBeLessThan(AUTO_WATERFALL_BACKGROUND.indexOf(luna));
+ expect(AUTO_WATERFALL[0]).toBe(luna);
+ expect(AUTO_WATERFALL_VISION[0]).toBe(luna);
+ expect(AUTO_WATERFALL_BACKGROUND[0]).toBe(luna);
});
});