YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
Make GPT-5.6 Luna the Auto default
Louis Beaumont5873eccd920b
diff --git a/packages/ai-gateway/src/handlers/chat.ts b/packages/ai-gateway/src/handlers/chat.tsindex d8ba3b9e7..897c455ec 100644--- a/packages/ai-gateway/src/handlers/chat.ts+++ b/packages/ai-gateway/src/handlers/chat.ts@@ -10,17 +10,14 @@ import { isFlexEligible } from '../utils/latency';import { routeTier, routerArm, TIER_HEAD } from './difficulty-router';import { captureException } from '@sentry/cloudflare';-// Auto model waterfall (INTERACTIVE) — leads with glm-5. Interactive is-// latency-bound (a user is waiting), and on these models intelligence == reasoning-// effort == latency: gemini-3.5-flash / gpt-5.x are only "smarter" at high effort,-// which is slow (3-10s+); at chat speed they drop BELOW glm-5's IQ. glm-5 is the-// fast/smart sweet spot for chat (~1.2s, AA 50, free Vertex MaaS). Smart reasoning-// models live on AUTO_WATERFALL_BACKGROUND where latency doesn't matter.+// Auto model waterfall (INTERACTIVE) — Luna leads while the OpenAI credit+// allocation is available. Keep fast/free models behind it for resilience if+// OpenAI is unavailable or rate-limited.// Exported so tests can pin that every chain entry has a MODEL_PRICING match// (otherwise served-model cost rows fall into the unknown-model estimate).export const AUTO_WATERFALL = [- 'glm-5', // fast (~1.2s) + AA 50, free Vertex MaaS — best fast/smart for latency-bound chat- 'gpt-5.6-luna', // paid high-volume fallback — stronger when the fast free primary is unavailable+ 'gpt-5.6-luna', // OpenAI-credit-funded default for Auto chat+ 'glm-5', // fast/free Vertex MaaS fallback'kimi-k2.5','glm-4.7','gemini-3-flash',@@ -28,29 +25,22 @@ export const AUTO_WATERFALL = [// Vision-capable models for requests containing imagesexport const AUTO_WATERFALL_VISION = [- 'gemini-3.5-flash', // multimodal, leads on agent/vision benchmarks- 'gpt-5.6-luna', // OpenAI vision fallback before the lower-capability free options+ 'gpt-5.6-luna', // OpenAI-credit-funded default for Auto vision chat+ 'gemini-3.5-flash', // multimodal fallback'llama-4-maverick', // free (Vertex MaaS), 400B MoE, strong vision + reasoning'gemini-3-flash', // near-free, good vision'llama-4-scout', // free (Vertex MaaS), 109B MoE, decent vision fallback'gemini-2.5-flash', // backup vision option];-// Background waterfall — for latency-tolerant traffic (pipes, daily summary,-// suggestions) where no user is waiting. Leads with gemini-3.5-flash on the FLEX-// tier: background is ~84% cache-reads, so flex's 0.1x cache discount makes it the-// cheapest decent-quality option ($/Mtok ≈ glm-5 but with the cache discount glm-5-// lacks). Measured (6/19): gpt-5.4 here cost $590/day vs $177 on flex for the SAME-// traffic — 3.3x more, on the small OpenAI pool, for a marginal IQ gain (54 vs ~50-55)-// that pipes don't need; reverted (#4285→). glm-5 + gemini-3-flash are standard-tier-// fallbacks for when flex is throttled. Luna is a paid, high-volume fallback-// immediately after the flex primary: it improves resilience without replacing-// the proven cache-efficient default for every scheduled run. Pi's streaming-// tool policy is forwarded by the OpenAI adapter, so this lane can use Luna for-// tool-enabled pipes as intended.+// Background waterfall — for pipes, summaries, and suggestions. Luna leads+// while OpenAI credits are available; Gemini FLEX and Vertex MaaS remain+// fallbacks for outages, quota limits, or later cost-control changes. Pi's+// streaming tool policy is forwarded by the OpenAI adapter, so Luna supports+// tool-enabled pipes on this lane.export const AUTO_WATERFALL_BACKGROUND = [- 'gemini-3.5-flash', // flex tier — cheapest on the cache-heavy background mix- 'gpt-5.6-luna', // paid fallback for complex / tool-enabled pipe work+ 'gpt-5.6-luna', // OpenAI-credit-funded default for Auto pipes+ 'gemini-3.5-flash', // flex-tier fallback'glm-5', // free Vertex MaaS fallback, standard tier'gemini-3-flash', // near-free safety net];diff --git a/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts b/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.tsindex 95c68ef68..88550d00e 100644--- a/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts+++ b/packages/ai-gateway/src/test/auto-cost-attribution.unit.test.ts@@ -73,10 +73,10 @@ describe('routing chains stay priceable (no silent $0.01 fallback rows)', () =>expect(getModelCost('gemini-3-flash', 1_000_000, 0)).toBeCloseTo(0.10, 10);});- it('places Luna behind the current auto primaries for text, vision, and pipes', () => {+ it('uses Luna as the current Auto primary for text, vision, and pipes', () => {const luna = 'gpt-5.6-luna';- expect(AUTO_WATERFALL.indexOf('glm-5')).toBeLessThan(AUTO_WATERFALL.indexOf(luna));- expect(AUTO_WATERFALL_VISION.indexOf('gemini-3.5-flash')).toBeLessThan(AUTO_WATERFALL_VISION.indexOf(luna));- expect(AUTO_WATERFALL_BACKGROUND.indexOf('gemini-3.5-flash')).toBeLessThan(AUTO_WATERFALL_BACKGROUND.indexOf(luna));+ expect(AUTO_WATERFALL[0]).toBe(luna);+ expect(AUTO_WATERFALL_VISION[0]).toBe(luna);+ expect(AUTO_WATERFALL_BACKGROUND[0]).toBe(luna);});});