Skip to content

armgit / fx0

Public Imported from GitHub
0

A lightweight design agent built with libfx. Describe a page and an agent writes it as one HTML file.

TypeScript

Count subagent model requests in their turn's 64 steps

Pranit75ae9d865d55
diff --git a/README.md b/README.md
index 88774c7..421a14c 100644
--- a/README.md
+++ b/README.md
@@ -16,7 +16,7 @@ Try it at [fx0.vercel.dev](https://fx0.vercel.dev), or clone it and make it your
- The agent can search the web and fetch pages for reference.
- Start from one of ten `design.md` styles, or upload your own. The ten styles, and the one-line descriptions in the picker, come from VoltAgent's [awesome-design-md](https://github.com/VoltAgent/awesome-design-md), which you can browse at [getdesign.md](https://getdesign.md). Thanks to VoltAgent for publishing them.
- Type a message before you sign in. The draft, images included, waits in the tab for 15 minutes while you sign in.
-- A turn stops after 64 model steps, which keeps an agent stuck in a loop from running up a bill.
+- A turn stops after 64 model steps, its subagents' included, which keeps an agent stuck in a loop from running up a bill.
- In paid-only chats, a text message sent while the agent works steers it, as in the fx CLI. Trial chats run each input as its own turn so free and paid funding stay separate.
## How fx0 uses libfx
@@ -95,7 +95,7 @@ fx0 keeps the transcript, designs and settings in Redis. Here is one message fro
A tool call the crash cut off runs again only if it's safe to repeat. libfx refuses a tool that doesn't say which it is, and fx0 marks its tools in [`lib/libfx-agent.ts`](lib/libfx-agent.ts): reading the design, publishing it, fetching a page, paging through a tool result and searching the tools run again. Edits, questions and subagents don't, so the model is told those calls may have run and decides what to do. The reads start as soon as the model asks for them, while libfx saves the call.
-A subagent is an in-memory `createFxEngine()` inside its tool call, billed to the same chat. It has the same design tools, except that it can't start a subagent of its own.
+A subagent is an in-memory `createFxEngine()` inside its tool call, billed to the same chat. It has the same design tools, except that it can't start a subagent of its own. Its model requests count toward the parent turn's 64 steps, in a count kept in Redis ([`lib/turn-budget.ts`](lib/turn-budget.ts)), so subagents can't multiply what one turn spends. A retried or recovered request counts once.
TL;DR:
@@ -171,6 +171,7 @@ npm run test:smoke # runs the production build with fake sessions
npm run test:journal:integration # needs KV_REST_API_URL/TOKEN, writes under its own prefix
npm run test:trial:integration # needs local redis-server/redis-cli; no remote services
npm run test:artifact:browser # needs local Chrome (or CHROME_PATH); checks design CSP enforcement
+npm run test:budget:integration # needs local redis-server/redis-cli; no remote services
```
The auth and route tests mock Vercel, AI Gateway and both databases, so check a real sign-in and real billing on a deployment.
diff --git a/lib/libfx-agent.ts b/lib/libfx-agent.ts
index 84b9720..3a4dc14 100644
--- a/lib/libfx-agent.ts
+++ b/lib/libfx-agent.ts
@@ -1,3 +1,4 @@
+import { createHash } from 'node:crypto'
import { createFxAgent, createFxEngine, type FxAgent, type FxAgentHostTool, type FxEffort, type FxHostTool, type FxSession, type FxToolContext } from 'libfx'
import { createTools, type ToolHost } from './tools'
import { gatewayFetch, type GatewayError, type DesignDraftObserver } from './gateway-session'
@@ -10,6 +11,7 @@ import { TRIAL_MODEL } from './trial-policy'
import { bindLibfxSession, chatOfLibfxSession, getActiveTurn } from './journal'
import { createDraftContext } from './draft-context'
import { closeDraftCall, storedDraftObserver } from './draft-store'
+import { turnBudget } from './turn-budget'
import {
DEFAULT_MODEL,
MAX_TURN_MODEL_STEPS,
@@ -36,9 +38,17 @@ export interface ChatRuntime {
toolHost(chatId: string, turnId: number, signal: AbortSignal): Promise<ToolHost>
}
-// Model requests of this turn so far, from the conversation the request carries:
-// the assistant messages after the last user message, plus this one. It needs no
-// state, so it holds across crashes and across invocations.
+// Which model request of a turn this is: who sent it, and the conversation it
+// carries. A retry or a crash's resend of a step carries the same conversation, so
+// it has the same name; every new step, even after compaction, has a new one.
+export function modelStepName(sender: string, body: unknown): string {
+ const prompt = (body as { prompt?: unknown } | null)?.prompt ?? null
+ return `${sender}:${createHash('sha256').update(JSON.stringify(prompt)).digest('base64url').slice(0, 22)}`
+}
+
+// Model requests of this conversation so far: the assistant messages after the
+// last user message, plus this one. It needs no state, so it holds even where the
+// turn's shared budget (turn-budget.ts) is unknown or lost.
export function turnModelStep(body: unknown): number {
const prompt = (body as { prompt?: unknown })?.prompt
if (!Array.isArray(prompt)) return 1
@@ -71,9 +81,12 @@ const chatGateways = new Map<string, ChatGateway>()
// Every model request uses the executing turn's recorded funding. With no usable
// credential the request fails: fx0 never falls back to the deployment's key.
// Public model metadata, which libfx reads without a session, goes out bare.
+// `sender` is the parent turn, or one subagent run: all of them share the turn's
+// MAX_TURN_MODEL_STEPS, so subagents cannot multiply what one turn spends.
function chatFetch(
chatId: () => Promise<string | null>,
executionTurn: () => number | undefined,
+ sender = 'parent',
): typeof fetch {
return async (input, init) => {
const turn = executionTurn()
@@ -81,24 +94,29 @@ function chatFetch(
const url = new URL(input instanceof Request ? input.url : String(input))
const method = (init?.method ?? (input instanceof Request ? input.method : 'GET')).toUpperCase()
const modelRequest = method === 'POST' && url.pathname.endsWith('/language-model')
+ let body: unknown = null
if (modelRequest && typeof init?.body === 'string') {
- let step = 1
- try { step = turnModelStep(JSON.parse(init.body)) } catch { /* libfx validates its own body */ }
- if (step > MAX_TURN_MODEL_STEPS) {
- log(id ?? 'unknown', 'turn:step-limit', { steps: MAX_TURN_MODEL_STEPS })
- return jsonError(400, STEP_LIMIT)
- }
+ try { body = JSON.parse(init.body) } catch { /* libfx validates its own body */ }
+ }
+ const limited = () => {
+ log(id ?? 'unknown', 'turn:step-limit', { steps: MAX_TURN_MODEL_STEPS, sender: sender === 'parent' ? 'parent' : 'subagent' })
+ return jsonError(400, STEP_LIMIT)
}
+ // A conversation that already shows the limit stops at once, with no stored state;
+ // the turn's shared budget below also counts its subagents and survives a crash.
+ if (modelRequest && turnModelStep(body) > MAX_TURN_MODEL_STEPS) return limited()
const gateway: ChatGateway = (id && chatGateways.get(id)) || { gatewaySession: null, rateLimitStreak: 0, lastGatewayError: null }
let observer: DesignDraftObserver | undefined
if (id) {
chatGateways.set(id, gateway)
- // Both reads use the same executing turn; reserving its preview adds no serial
- // round trip to the credential lookup before the model request starts.
- const [funding, draft] = await Promise.all([
+ // All three use the same executing turn; reserving its preview and its step adds
+ // no serial round trip to the funding lookup before the model request starts.
+ const [funding, draft, steps] = await Promise.all([
readExecutionFunding(id, turn),
turn && modelRequest ? storedDraftObserver(id, turn).catch(() => undefined) : undefined,
+ turn && modelRequest ? turnBudget().reserve(id, turn, modelStepName(sender, body)) : 0,
])
+ if (steps > MAX_TURN_MODEL_STEPS) return limited()
// The request fails below with "sign in again"; the log says why, with
// the turn and reason only, never a credential.
if (!funding.ok && modelRequest) log(id, 'funding:refused', { turn: turn ?? null, reason: funding.refused })
@@ -226,8 +244,9 @@ export async function chatSession(
}
// A subagent is an ordinary in-memory engine inside its tool call, billed to the
-// same chat. A crash mid-call is the parent's to recover: subagent is not
-// idempotent, so libfx tells the model the call may have run.
+// same chat and counted in the parent turn's steps. A crash mid-call is the
+// parent's to recover: subagent is not idempotent, so libfx tells the model the
+// call may have run; a run the model starts again counts as a new one.
export async function runSubagent(
runtime: ChatRuntime,
host: SessionState,
@@ -242,7 +261,7 @@ export async function runSubagent(
const pinned = active?.id === turnId ? active.settings : undefined
const engine = await createFxEngine({
apiKey: 'session-authenticated',
- fetch: chatFetch(async () => chatId, () => turnId),
+ fetch: chatFetch(async () => chatId, () => turnId, `subagent:${crypto.randomUUID()}`),
model: pinned
? { id: pinned.model, effort: pinned.effort as FxEffort, fast: pinned.fast }
: chatModel(host, funding.ok && 'kind' in funding.credential),
diff --git a/lib/turn-budget.ts b/lib/turn-budget.ts
new file mode 100644
index 0000000..283cf29
--- /dev/null
+++ b/lib/turn-budget.ts
@@ -0,0 +1,34 @@
+// @ts-expect-error Node strip-types tests require the source extension.
+import { prefixed, redis } from './journal.ts'
+
+// A turn's model requests, its subagents' included, counted against the turn's
+// step limit. Each request is named by who sent it and what conversation it
+// carries (see modelStepName): a retry or a crash's resend counts once, and every
+// real step counts again. One script adds the name and counts, so two subagents
+// running at once cannot both take the last step.
+const RESERVE = `
+redis.call('HSETNX', KEYS[1], ARGV[1], 1)
+redis.call('EXPIRE', KEYS[1], ARGV[2])
+return redis.call('HLEN', KEYS[1])
+`
+// Longer than any turn runs, so a recovered turn keeps what it already spent.
+const TTL_SECONDS = 24 * 60 * 60
+
+interface TurnBudgetRedis {
+ eval(script: string, keys: string[], args: (string | number)[]): Promise<unknown>
+}
+
+export function createTurnBudget(client: TurnBudgetRedis, prefix: string) {
+ return {
+ // How many model requests the turn has made, counting this one.
+ async reserve(chatId: string, turnId: number, step: string): Promise<number> {
+ const count = Number(await client.eval(RESERVE, [`${prefix}:${chatId}:${turnId}`], [step, TTL_SECONDS]))
+ if (!Number.isSafeInteger(count) || count < 1) throw new Error('Invalid turn step count')
+ return count
+ },
+ }
+}
+
+export function turnBudget() {
+ return createTurnBudget(redis(), prefixed('turn-steps'))
+}
diff --git a/package.json b/package.json
index 187cd92..c70d9d0 100644
--- a/package.json
+++ b/package.json
@@ -15,6 +15,7 @@
"test:smoke": "node tests/local-auth-smoke.mjs",
"db:migrate": "node --env-file-if-exists=.env.local scripts/migrate-database.mjs",
"test:artifact:browser": "node --experimental-strip-types --test tests/design-artifact.browser.mjs",
+ "test:budget:integration": "node --experimental-strip-types --test tests/turn-budget.integration.mjs",
"test:trial:integration": "node --experimental-strip-types --test tests/trial.integration.mjs",
"test:credential:integration": "node --experimental-strip-types --test tests/turn-credential.integration.mjs"
},
diff --git a/tests/libfx-agent.test.mjs b/tests/libfx-agent.test.mjs
index 5e97603..ffed225 100644
--- a/tests/libfx-agent.test.mjs
+++ b/tests/libfx-agent.test.mjs
@@ -12,6 +12,7 @@ const fakes = {
'./journal': dataUrl(forward('journal', ['appendEvent', 'bindLibfxSession', 'chatOfLibfxSession', 'getActiveTurn'])),
'./draft-store': dataUrl(forward('drafts', ['storedDraftObserver','closeDraftCall'])),
'./turn-credential': dataUrl(forward('credentials', ['readTurnCredential', 'readExecutionFunding'])),
+ './turn-budget': dataUrl(forward('budget', ['turnBudget'])),
}
registerHooks({ resolve(specifier, context, next) {
if (context.parentURL?.includes('/lib/')) {
@@ -20,17 +21,28 @@ registerHooks({ resolve(specifier, context, next) {
}
return next(specifier, context)
} })
-const { durableAgent, runSubagent, turnModelStep, STEP_LIMIT, SIGN_IN_AGAIN } = await import('../lib/libfx-agent.ts')
+const { durableAgent, modelStepName, runSubagent, turnModelStep, STEP_LIMIT, SIGN_IN_AGAIN } = await import('../lib/libfx-agent.ts')
const { MAX_TURN_MODEL_STEPS } = await import('../lib/session.ts')
const MODEL_URL = 'https://ai-gateway.vercel.sh/v3/ai/language-model'
const stored = new Map()
const executions = new Map()
+// Each turn's step names, as tests/turn-budget.integration.mjs checks the real script keeps them.
+const turnSteps = new Map()
let agentOptions = null
globalThis.__fx0Agent = {
libfx: { createFxAgent: (options) => { agentOptions = options; return {} }, createFxEngine: async () => ({}) },
+ budget: {
+ turnBudget: () => ({
+ async reserve(chatId, turnId, step) {
+ const key = `${chatId}:${turnId}`
+ if (!turnSteps.has(key)) turnSteps.set(key, new Set())
+ return turnSteps.get(key).add(step).size
+ },
+ }),
+ },
journal: {
- chatOfLibfxSession: async (id) => ({ wrun_a: 'chat-a', wrun_b: 'chat-b' })[id] ?? null,
+ chatOfLibfxSession: async (id) => ({ wrun_a: 'chat-a', wrun_b: 'chat-b', wrun_c: 'chat-c' })[id] ?? null,
bindLibfxSession: async (_chat, id) => id,
emitLive: () => {},
getActiveTurn: async () => null,
@@ -223,3 +235,58 @@ test('Gateway\'s own reason survives a rejection, scrubbed of the credential', a
assert.equal(error.type, 'insufficient_credits')
assert.ok(!error.message.includes('token-secret-a'))
})
+
+test('a model request is named by its sender and conversation, so only a resend shares a name', () => {
+ const body = { prompt: [{ role: 'user', content: [{ type: 'text', text: 'go' }] }] }
+ assert.equal(modelStepName('parent', body), modelStepName('parent', JSON.parse(JSON.stringify(body))))
+ assert.notEqual(modelStepName('parent', body), modelStepName('subagent:a', body))
+ assert.notEqual(modelStepName('parent', body), modelStepName('parent', { prompt: [...body.prompt, { role: 'assistant', content: [] }] }))
+ // Compaction can shorten a conversation to a length seen before; its content differs.
+ assert.notEqual(modelStepName('parent', { prompt: [{ role: 'user', content: 'summary' }] }), modelStepName('parent', { prompt: [{ role: 'user', content: 'go' }] }))
+ assert.equal(modelStepName('parent', {}), modelStepName('parent', null))
+ assert.match(modelStepName('parent', body), /^parent:[A-Za-z0-9_-]{22}$/)
+})
+
+test('a turn\'s subagents spend its 64 model requests with it, and a retried request counts once', async (t) => {
+ stored.set('chat-c', credential('token-c'))
+ t.after(() => { stored.delete('chat-c'); turnSteps.clear() })
+ agentOptions.onEvent({ type: 'acp.send', message: { method: 'session/prompt', params: { sessionId: 'wrun_c', _meta: { fx: { turnId: 'm51' } } } } })
+ const network = t.mock.method(globalThis, 'fetch', async () => Response.json({}))
+ // Each step's conversation is the last one plus the model's reply and its tool's result.
+ const conversation = (steps, text = 'go') => [
+ { role: 'user', content: [{ type: 'text', text }] },
+ ...Array.from({ length: steps }, (_, i) => [{ role: 'assistant', content: [{ type: 'text', text: `step ${i}` }] }, { role: 'tool', content: [] }]).flat(),
+ ]
+ for (let step = 0; step < 40; step++) assert.equal((await modelRequest('wrun_c', conversation(step))).status, 200, `parent step ${step + 1}`)
+ // libfx retrying a step, or resending it after a crash, sends the same conversation.
+ for (let retry = 0; retry < 3; retry++) assert.equal((await modelRequest('wrun_c', conversation(39))).status, 200)
+ assert.equal(turnSteps.get('chat-c:51').size, 40)
+
+ // Subagents of the same turn, each running its own conversation until refused.
+ const runs = []
+ t.mock.method(globalThis.__fx0Agent.libfx, 'createFxEngine', async (options) => ({
+ prompt: () => {
+ const result = (async () => {
+ const statuses = []
+ for (let step = 0; step < 30; step++) {
+ const response = await options.fetch(MODEL_URL, { method: 'POST', headers: { 'ai-language-model-id': options.model.id }, body: JSON.stringify({ prompt: conversation(step, 'subtask') }) })
+ statuses.push(response.status)
+ if (response.status !== 200) break
+ }
+ runs.push(statuses)
+ return { stopReason: 'end_turn' }
+ })()
+ return { result, async *[Symbol.asyncIterator]() { await result; yield { type: 'text_delta', delta: 'Child done.' } } }
+ },
+ close: async () => {},
+ }))
+ const subagent = () => runSubagent({ toolHost: async () => ({}) }, { id: 'chat-c' }, 51, 'instructions', 'task', new AbortController().signal)
+ await subagent()
+ await subagent()
+ // The first takes the 24 requests left; a second run's same conversations are its own, and none are left.
+ assert.deepEqual(runs.map((statuses) => [statuses.filter((status) => status === 200).length, statuses.at(-1)]), [[24, 400], [0, 400]])
+ const parent = await modelRequest('wrun_c', conversation(40))
+ assert.equal(parent.status, 400)
+ assert.equal((await parent.json()).error.message, STEP_LIMIT)
+ assert.equal(network.mock.callCount(), 40 + 3 + 24, 'nothing past the limit reaches the network')
+})
diff --git a/tests/turn-budget.integration.mjs b/tests/turn-budget.integration.mjs
new file mode 100644
index 0000000..4528165
--- /dev/null
+++ b/tests/turn-budget.integration.mjs
@@ -0,0 +1,46 @@
+import assert from 'node:assert/strict'
+import { execFile, spawn } from 'node:child_process'
+import { mkdtemp, rm } from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { promisify } from 'node:util'
+import test from 'node:test'
+
+// The turn step budget's real Lua on a local redis-server; no remote services.
+const run = promisify(execFile)
+const { createTurnBudget } = await import('../lib/turn-budget.ts')
+
+test('a turn counts each step once across its parent and subagents, even when they run at once', async (t) => {
+ const dir = await mkdtemp(join(tmpdir(), 'fx0-turn-budget-'))
+ const socket = join(dir, 'redis.sock')
+ const server = spawn('redis-server', ['--port', '0', '--unixsocket', socket, '--save', '', '--appendonly', 'no'], { stdio: 'ignore' })
+ t.after(async () => { server.kill(); await new Promise(resolve => server.once('exit', resolve)); await rm(dir, { recursive: true, force: true }) })
+ for (let i = 0; i < 100; i++) {
+ try { await run('redis-cli', ['-s', socket, 'PING']); break } catch { await new Promise(resolve => setTimeout(resolve, 10)) }
+ }
+ const redisCli = (...args) => run('redis-cli', ['-s', socket, ...args.map(String)])
+ const evalClient = {
+ async eval(script, keys, args) {
+ const { stdout } = await redisCli('--json', 'EVAL', script, keys.length, ...keys, ...args)
+ return JSON.parse(stdout)
+ },
+ }
+ const budget = createTurnBudget(evalClient, 'budget-test')
+
+ // 70 different steps at once, from the parent and two subagents: each gets its own count.
+ const names = Array.from({ length: 70 }, (_, i) => `${['parent', 'subagent:a', 'subagent:b'][i % 3]}:step-${i}`)
+ const counts = await Promise.all(names.map((name) => budget.reserve('chat', 7, name)))
+ assert.deepEqual([...counts].sort((a, b) => a - b), Array.from({ length: 70 }, (_, i) => i + 1))
+
+ // Resending a step, even many at once, never raises the count.
+ const resent = await Promise.all(Array.from({ length: 10 }, () => budget.reserve('chat', 7, names[0])))
+ assert.deepEqual(resent, Array(10).fill(70))
+
+ // Another turn, or another chat, has its own budget.
+ assert.equal(await budget.reserve('chat', 8, names[0]), 1)
+ assert.equal(await budget.reserve('other-chat', 7, names[0]), 1)
+
+ // The count outlives any turn, then expires.
+ const ttl = Number((await redisCli('TTL', 'budget-test:chat:7')).stdout)
+ assert.ok(ttl > 86_000 && ttl <= 86_400, `ttl ${ttl}`)
+})