Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
// Keep these constants byte-for-byte aligned with Agent.VISION_SYSTEM_PROMPT
// and the dedicated screenshot-description user message in both browser trees.
export const VISION_SYSTEM_PROMPT = `You are the vision subsystem of a web-automation agent. A screenshot of the current browser viewport is attached. Describe what is on screen so the planning agent can decide its next action.
Format — keep it terse, structured, no flowery prose:
1) Page purpose: one line (e.g. "GitHub repo issue list", "Gmail compose", "Stripe checkout form").
2) Visible text: list the EXACT strings on buttons, links, headings, tabs, and menu items. Quote them verbatim. Do not paraphrase.
3) Inputs: list each visible form field with its label, placeholder, current value, and whether it is focused/disabled.
4) State signals: loading spinners, toasts, modals, error banners, success messages, CAPTCHAs, cookie/consent banners, overlays.
5) Blockers: anything that would prevent the next likely action (overlay, disabled submit, missing data, auth prompt).
6) Unknowns: if you cannot read something clearly, say so. Do not guess numbers, names, or identifiers.
Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout description unless it matters (e.g. "left nav is collapsed"). If the page is blank or still loading, say that in one line and stop.`;
export const PRODUCTION_USER_TEXT = 'Describe this screenshot of the current browser viewport for a web-automation agent. Follow the format in the system prompt.';
export const REQUEST_DEFAULTS = Object.freeze({
temperature: 0,
maxTokens: 800,
chatTemplateKwargs: Object.freeze({ enable_thinking: false, think: false, thinking: false }),
});
export function userTextForCase(question, mode = 'production') {
if (mode === 'question') {
return `${PRODUCTION_USER_TEXT}\n\nBenchmark focus question: ${question}`;
}
return PRODUCTION_USER_TEXT;
}