Skip to content

Lightpanda: the headless browser designed for AI and automation

removed in favor of https://lightpanda.io/docs/usage/agent

Pierre Tachoireb003614064f9
diff --git a/docs/agent.md b/docs/agent.md
deleted file mode 100644
index 9a82187ca..000000000
--- a/docs/agent.md
+++ /dev/null
@@ -1,530 +0,0 @@
-# Agent mode
-
-`lightpanda agent` lets you drive a headless browser by talking to it.
-
-You tell it where to go and what to extract, in plain English or with slash
-commands, and it controls a real browser to do the work. Think of it as a
-robot you're directing to use the web, not a chatbot you're having a
-conversation with.
-
-Every session starts by navigating to a page, either by
-saying so ("go to news.ycombinator.com") or by typing `/goto <url>`. There's
-no window to look at; the browser runs headlessly and you see its output
-(extracted data, the agent's answer) in your terminal.
-
-## How to think about it
-
-The agent stacks three layers:
-
-1. **The browser.** Loads pages, runs JavaScript, tracks cookies, follows
- redirects. The same engine that powers `lightpanda serve` and
- `lightpanda fetch`.
-2. **A set of tools.** Things the browser knows how to do: `goto`, `click`,
- `fill`, `extract`, `evaluate`, `search`, and more. Each is available as a
- slash command (`/goto`, `/click`, ...).
-3. **An LLM.** Reads your plain-English request and decides which tools to
- call. Optional. The agent can also run without it.
-
-You can talk to any of the three layers. Type `/goto example.com` and you
-call the browser tool directly. Type "find me the cheapest flight from NYC
-to Tokyo" and the LLM picks the tools to use. Save what worked to a `.js`
-file and you can replay the whole flow later without ever calling the LLM
-again.
-
-## Quick start
-
-Set an API key for your preferred provider:
-
-```bash
-export ANTHROPIC_API_KEY=sk-ant-...
-```
-
-(Or `OPENAI_API_KEY` / `GOOGLE_API_KEY`. Without a key the agent runs in a
-slash-commands-only mode without natural language.)
-
-Launch the REPL:
-
-```bash
-./lightpanda agent
-```
-
-Tell it what you want:
-
-```
-❯ go to news.ycombinator.com and get me the top story title and points
-```
-
-The agent navigates, extracts, prints the answer. When you're happy with
-the result, save it and replay it right away to check it holds up without
-the LLM:
-
-```
-❯ /save hn-top-story.js
-❯ /load hn-top-story.js
-❯ /quit
-```
-
-You now have a `.js` file that does the same job deterministically:
-
-```bash
-./lightpanda agent hn-top-story.js
-```
-
-No LLM call, no API key needed at replay time, sub-second to run.
-
-For a longer worked example (logging into a site, extracting structured
-data across pages), see the [tutorial](agent-tutorial.md).
-
-### Other ways to launch
-
-```bash
-# Force a specific provider, ignoring auto-detection
-./lightpanda agent --provider anthropic
-
-# REPL without an LLM: /goto, /extract, ... work; plain English doesn't
-./lightpanda agent --no-llm
-
-# One-shot: ask a question, print the answer to stdout, exit
-./lightpanda agent --task "what is on the front page of hn?"
-```
-
-## Tips for getting useful saved scripts
-
-The whole value of a saved script is that it keeps working after you walk away.
-
-- **Ask for the data you want, not the conversation about it.** "Get me the
- top 5 HN story titles and points, print them to stdout" beats "show me
- what's on Hacker News today." The synthesizer captures the data you asked
- for as a `page.extract()` call; vague prompts produce vague recordings.
-- **Be specific about the page.** "Go to news.ycombinator.com" is much
- better than "find me what's on Hacker News".
-- **Check the file with `cat your-script.js` before running it.** The
- script holds the *calls*, not the data, so look for a `page.extract(...)`
- whose schema covers the fields you asked for. If there isn't one, the
- recording missed something — try rewording your prompt.
-- **When the page changes and a saved script breaks**, re-run with the LLM,
- get an updated answer, save again. You only pay the LLM when something
- genuinely changed.
-
-## Providers and API keys
-
-The agent needs an LLM to interpret natural language. Set the relevant API
-key as an environment variable, or pass `--provider` explicitly.
-
-| Provider | Flag | API key env |
-|-----------|------------------------|--------------------------------------|
-| Anthropic | `--provider anthropic` | `ANTHROPIC_API_KEY` |
-| OpenAI | `--provider openai` | `OPENAI_API_KEY` |
-| Gemini | `--provider gemini` | `GOOGLE_API_KEY` or `GEMINI_API_KEY` |
-| Ollama | `--provider ollama` | none (local) |
-
-`--model` falls back to a sensible per-provider default. `--base-url`
-overrides the API endpoint (Ollama defaults to `http://localhost:11434/v1`).
-`--list-models` prints what the resolved provider offers and exits.
-`--system-prompt` swaps in your own system prompt. `--verbosity
-<low|medium|high>` tunes how much progress detail goes to stderr (`--task`
-defaults to `low`, escalating to `high` when stderr is piped or redirected
-so harnesses capture the full `[tool/result]` trace).
-
-### How a provider gets picked
-
-Without `--provider`, the agent picks one in this order:
-
-1. **Remembered** - whatever you last selected with `/provider` or `/model`,
- persisted per-directory in `.lp-agent.zon`, as long as its key is still
- set.
-2. **Auto-detected** - the first key found in priority order
- (`ANTHROPIC_API_KEY` → `GOOGLE_API_KEY`/`GEMINI_API_KEY` →
- `OPENAI_API_KEY`). With several keys on a TTY, you'll be prompted to
- pick.
-3. **Local Ollama** - if no cloud key is set, the agent probes
- `http://localhost:11434/v1` and uses it if there's at least one model
- pulled.
-4. **No provider at all** - falls back to the basic REPL (slash commands
- only). Natural language and LLM-driven commands (`/login`, `/logout`,
- `/acceptCookies`) will reject.
-
-`--no-llm` is the explicit override. It forces the basic REPL even when an
-API key is present. Useful for testing slash commands without burning
-tokens.
-
-### Reasoning effort
-
-`--effort <none|minimal|low|medium|high|xhigh>` sets the per-turn reasoning
-budget for thinking models. It maps to each provider's native
-reasoning-effort knob and is ignored by non-thinking models. The REPL
-defaults to `low` so turns stay snappy. `--task` defaults to `medium`
-where answer quality matters more than per-turn latency. Higher
-effort can mean fewer tool calls per task (the model plans better), so it's
-a real tradeoff rather than a pure slowdown. Change it live in the REPL
-with `/effort`; selection persists in `.lp-agent.zon`. Script replay makes
-no LLM calls, so effort doesn't apply there.
-
-The default depends on how you launched, which is easy to miss:
-
-| Context | Default effort | Why |
-|----------------|----------------|--------------------------------------|
-| REPL | `low` | Keeps turns snappy |
-| `--task` | `medium` | Answer quality matters more |
-
-## Slash commands
-
-The REPL uses a small slash-command language for browser actions. Each line
-you type at the prompt is either a slash command, a `#` comment, a blank
-line, or (when an LLM is configured) a natural-language prompt.
-
-Slash commands accept:
-
-- A single positional value, when the tool has exactly one required field.
- `/goto 'https://example.com'`. The value can itself be quoted JSON when
- that's what the field takes: `/extract '{"karma":"#karma"}'` passes the
- string to extract's one required field, `schema`.
-- `key=value` pairs. Values may be bare or quoted; strings with whitespace
- must be quoted. `/fill selector='#email' value='[email protected]'`. A positional
- and `key=value` pairs can be mixed, but the positional must come first:
- `/extract '{"karma":"#karma"}' save=me`.
-- A raw `{json}` blob, handed straight to the tool.
- `/findElement {"role":"button"}`.
-
-Tools whose selector is optional (`/click`, `/hover`, `/findElement`) take
-no positional and must use `key=value` form: `/click selector='a.login'`.
-`/help <tool>` prints any tool's JSON schema — required fields, optional
-fields, and types.
-
-Quoting is content-aware: `'…'`, `"…"`, and triple-quoted `'''…'''` /
-`"""…"""` for values that mix quote styles or span multiple lines. Paste a
-multi-line command and the REPL keeps the whole paste as one input; typed
-line by line, Enter submits at each newline.
-
-### LLM-driven helpers
-
-Three slash commands trigger an LLM turn rather than a direct tool call:
-
-| Command | What it does |
-|------------------|-------------------------------------------------------|
-| `/login` | Fills credentials from `$LP_*` env vars. |
-| `/logout` | Finds the logout control and signs out. |
-| `/acceptCookies` | Dismisses the consent banner. |
-
-All three require an LLM. `--no-llm` rejects them.
-
-### Meta commands
-
-These don't drive the browser, they control the REPL itself:
-
-| Command | What it does |
-|--------------------------|----------------------------------------------------|
-| `/help` | Lists tools. `/help <tool>` prints the JSON schema.|
-| `/provider [name]` | Lists or switches provider. |
-| `/model [name]` | Lists or switches model for the active provider. |
-| `/effort <level>` | Sets reasoning budget. Saved to `.lp-agent.zon`. |
-| `/verbosity <level>` | Tunes the log level. Levels: low, medium, high. |
-| `/usage` | Prints cumulative token usage and cache hit rate. |
-| `/save [file] [prompt]` | Writes the session to a script. `.js` is appended to the name if missing; trailing words guide the synthesizer.|
-| `/load <path>` | Runs a script from disk against the current session.|
-| `/clear` | Forgets the conversation (history, usage, recorded actions, node IDs); keeps the page and cookies.|
-| `/reset` | Full reset: everything `/clear` does, plus a fresh browser session, dropping the page, cookies, and storage.|
-| `/quit` | Exits the REPL. |
-
-Meta commands are never recorded.
-
-Use `/clear` when you want to test a new prompt against the current
-page without losing your login or cookies. Use `/reset` when you need
-a completely clean browser (no cookies, no current page, no storage).
-
-## REPL features
-
-- **Ghost hints.** There's no separate status line; guidance renders as dim
- ghost text after the cursor and disappears as you type over it. It
- previews the rest of the first matching command name, the argument shape
- of the tool you're typing (`/evaluate ` shows
- `<script> [url=…] [timeout=…] [save=…]`), and contextual nudges like
- "press Ctrl-D again to exit".
-- **JS mode (`!`).** Type `!` on an empty prompt to toggle a scratchpad
- where the whole line runs as page-side JavaScript, same context as
- `/evaluate` so `document` and `window` are in scope. The prompt switches
- to `!` with a "JS mode - esc to exit" ghost hint. Handy for poking at
- a page without wrapping every line in `/evaluate`:
-
- ```
- ❯ !
- ! document.title
- "Hacker News"
- ! document.querySelectorAll('tr.athing').length
- 30
- ```
-
- `$LP_*` refs are still resolved at execution, console output is echoed
- back, and `Esc` exits. JS-mode lines are not recorded.
-- **Tab completion** (case-insensitive). Cycles through `/<tool>` and meta
- slash commands. The dim grey suffix shown after the cursor is the first
- match.
-- **Persistent history.** Stored in `.lp-history` in the working directory.
-- **Stdout vs stderr.** The final assistant answer and data-producing slash
- commands (`/extract`, `/evaluate`, `/markdown`, `/tree`, ...) write to
- stdout. Tool calls, progress, and errors go to stderr. So
- `lightpanda agent --task ... > out.txt` captures a clean answer.
-
-## Extracting data
-
-`/extract` takes a JSON schema where each value tells the extractor what to
-lift off the page. The result is printed to stdout as a single JSON object.
-
-```console
-# The demo shop renders its product data with an XHR after the initial
-# HTML, so wait for it to land before extracting.
-/goto 'https://demo-browser.lightpanda.io/campfire-commerce/'
-/waitForSelector '#product-features li'
-/extract '{"name": "#product-name", "price": "#product-price", "related": [{"selector": "#product-related div", "fields": {"name": "h4", "price": "p"}}]}'
-```
-
-```json
-{
- "name": "Outdoor Odyssey Nomad Backpack 60 liters",
- "price": "$244.99",
- "related": [
- { "name": "Outdoor Odyssey Hiking Poles", "price": "$79.99" },
- { "name": "Outdoor Odyssey Sleeping Bag", "price": "$129.99" },
- { "name": "Outdoor Odyssey Water Bottle", "price": "$19.99" }
- ]
-}
-```
-
-Supported value forms:
-
-- `"<sel>"`: `textContent.trim()` of the first match.
-- `""`: the matched element's own text (only inside a `fields` block).
-- `["<sel>"]`: text of every match. Sugar for `[{"selector": "<sel>"}]`.
-- `{"selector": "<sel>", "attr": "<name>"}`: attribute of the first match.
-- `[{"selector": "<sel>", "fields": {…}}]`: array of records, each
- `fields` value resolved relative to the matched element.
-
-A selector that matches nothing yields a null or empty field, not an error.
-That's deliberate, but it means a stale selector fails quietly. If a run
-comes back with blanks where you expected data, suspect the selectors or a
-missing wait before you suspect the page.
-
-The schema is parsed in Zig before the page-side walker runs, so malformed
-schemas are rejected up front with a plain `Error: InvalidParams` rather
-than a V8 stack trace.
-
-## Cross-call state with `lp.*`
-
-`/extract` and `/evaluate` each return one value per call. To carry data
-between calls (capture a list on one page, walk it across navigations) use
-the `save=` modifier:
-
-```console
-/goto 'https://news.ycombinator.com/'
-
-/extract '''
-{
- "stories": [{
- "selector": "tr.athing",
- "limit": 5,
- "fields": {
- "id": {"attr": "id"},
- "title": ".titleline > a"
- }
- }]
-}
-''' save=front
-
-/evaluate '''
-console.log(lp.front.stories[0].title);
-'''
-```
-
-`save=<name>` stashes the result keyed by `<name>` in a session-scoped
-store instead of printing it. Every subsequent `/evaluate` sees it as
-`globalThis.lp.<name>`.
-
-Any mutation of `lp.*` inside `/evaluate` is persisted at the end of the
-call. Adding (`lp.x = …`), updating
-(`lp.front.stories[0].comments = […]`), or deleting (`delete lp.x`) all
-propagate. The next `/evaluate` sees the update, even after a navigation,
-because the store lives session-side, not on the page.
-
-The store is **script-run scoped**: bound to the session that runs the
-script, gone when that session ends. There's no cross-session persistence;
-if you need that, use `localStorage` (origin-scoped, persists within a
-session).
-
-## JavaScript scripts
-
-`./lightpanda agent script.js` runs a script without any LLM call. Scripts
-are plain JavaScript plus the `Page` object — the only global. `new Page()`
-makes a page, `await page.goto(url)` navigates it, and every other primitive
-is a synchronous method on it:
-
-```js
-const page = new Page();
-await page.goto("https://example.com");
-page.click({ selector: "a.login" });
-page.evaluate("document.title");
-```
-
-`page.goto(...)` is **async — always `await` it**. Every other page method is
-**synchronous**: write `const data = page.extract(…)`, not
-`await page.extract(…)`. (`page.evaluate(...)` can run async JS inside the
-page, but the call itself still returns synchronously.) The script body runs
-as an async function, so top-level `await` is allowed. Re-navigating reuses
-the same page object: a later `await page.goto(url2)` keeps `page` valid and
-points it at the new URL.
-
-It's not Node.js. There's no `require`, `process`, `fs`, npm package
-loading, or Node standard library. The `page.evaluate(...)` method runs its
-string in the current page context; page scripts can't see agent variables
-or agent primitives.
-
-Whatever the script `return`s is printed automatically, so a script that ends
-with `return page.extract({...})` will print the extraction result to stdout.
-A bare trailing expression is not printed. Tool errors throw JavaScript
-exceptions and stop execution.
-
-See [agent-script.md](agent-script.md) for the full script format reference.
-
-### Saving and loading
-
-`/save [file.js]` writes the current session to a `.js` file. `/load <path>`
-runs a script from disk against the current session.
-
-`/save` arguments are positional: the first token is the filename (`.js`
-is appended when missing, so `/save test` writes `test.js`; quote names
-with spaces) and anything after it is passed to the synthesizer as
-guidance — `/save hn.js keep only the extraction`. With no argument at
-all, the file gets an auto-generated `session-*.js` name.
-
-`/save` works one of two ways, decided by the session rather than by any
-argument — if the REPL has an LLM you get synthesis, with `--no-llm` (or
-no resolved provider) you get the deterministic transcription:
-
-- **With `--no-llm`** it transcribes the session deterministically.
- State-mutating commands (`/goto`, `/click`, `/fill`, `/scroll`, `/hover`,
- `/selectOption`, `/setChecked`, `/waitForSelector`, `/waitForScript`,
- `/waitForState`, `/press`, `/evaluate`, `/extract`) become JavaScript
- calls. Read-only
- commands (`/tree`, `/markdown`, `/links`, `/findElement`, ...) are
- dropped. Each natural-language prompt that produced recorded actions is
- written as a `// <prompt>` comment above its calls so the script stays
- readable.
-- **With an LLM** it synthesizes an idiomatic script from the whole
- session. The synthesis prompt asks for JavaScript only ("no commentary"),
- so the result generally has no such comments: the model folds intent
- into the code and drops dead-ends. Returned data is whatever the script
- `return`s, which prints automatically on replay.
-
-## One-shot mode (`--task`)
-
-```console
-./lightpanda agent --provider gemini \
- --task "what is the top story on news.ycombinator.com?"
-```
-
-`--task` runs a single user turn, prints the final answer to stdout, and
-exits. On a TTY a spinner on stderr shows the tool currently running while
-the agent works; raise `--verbosity` for the full `[tool/result]` trace.
-
-Combine with `-a <path>` / `--attach <path>` (repeatable) to feed
-local files to providers that accept attachments — for example, a list of
-items to look up, or a document to cross-check against a live page:
-
-```console
-./lightpanda agent -a invoice.pdf \
- --task "open shop.example.com/orders/1042 and check the order total matches the attached invoice"
-```
-
-Text files are inlined
-into the prompt (max 512 KiB each); binary files (image, audio, pdf) are
-base64-encoded inline (max 20 MiB each). Unsupported MIME types fail before
-any browser work runs.
-
-`--task` conflicts with the positional script argument.
-
-## Driving Lightpanda from an external LLM agent
-
-When the calling agent already has its own LLM (e.g. Claude Code), use
-`lightpanda mcp` rather than `lightpanda agent`. The MCP server exposes the
-same browser tools listed below, so the external agent does the planning
-while Lightpanda only drives the browser. No `--provider` or API key is
-required on the Lightpanda side.
-
-```json
-{
- "mcpServers": {
- "lightpanda": {
- "command": "/path/to/lightpanda",
- "args": ["mcp"]
- }
- }
-}
-```
-
-Tool names are camelCase and case-sensitive. MCP clients must call the
-canonical tags (`goto`, `evaluate`, `tree`, `save`, ...).
-
-For sub-task delegation in the other direction (calling Lightpanda's own
-LLM-driven agent in a one-shot fashion), use `--task` on stdin.
-
-### Saving a script over MCP
-
-`lightpanda mcp` exposes a `save` tool so an external agent can persist the
-session as a `.js` script for later deterministic replay. Unlike the
-standalone agent's `/save`, the MCP server has no LLM of its own, so the
-calling client holds the conversation and synthesizes the script itself.
-
-| Tool | Args | Effect |
-|--------|------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
-| `save` | `{ path: string, script: string }` | Write `script` to `path` (relative, no `..`; created or overwritten) and return the absolute location and line count. |
-
-The tool's description carries the same synthesis guidance the agent's
-`/save` gives its LLM: drive a `Page` object (`await page.goto(...)`,
-`page.click(...)`, `page.fill(...)`, `page.extract(...)`, ...) as JavaScript
-calls, drop dead-ends, keep `$LP_*`
-placeholders. Any literal `LP_*` value is scrubbed back to its placeholder
-before the file is written. The result runs without an LLM via
-`./lightpanda agent session.js`.
-
-## Browser tools
-
-The agent and MCP server share the tool set defined in
-`src/browser/tools.zig`. Highlights:
-
-- `goto`, `search` (Tavily when `TAVILY_API_KEY` is set, DuckDuckGo
- otherwise)
-- `tree`, `markdown`, `html`, `links`, `interactiveElements`,
- `structuredData`, `detectForms`, `nodeDetails`, `findElement`
-- `click`, `fill`, `hover`, `press`, `scroll`, `selectOption`, `setChecked`,
- `waitForSelector`, `waitForScript`, `waitForState`
-- `extract` (the schema-driven data tool), `evaluate`, `consoleLogs`,
- `getUrl`, `getCookies`, `getEnv`
-
-Selectors prefer CSS over `backendNodeId` for the click-family tools since
-node IDs are invalidated by any DOM mutation. The system prompt enforces
-this for the LLM.
-
-## Security notes
-
-- The agent treats page content as untrusted data, not instructions. URLs
- surfaced by a page are not followed unless they match the user's task.
-- `$LP_*` environment variable references in `/fill` values are resolved
- at execution time inside the subprocess, so credentials never enter the
- LLM context. Conventional naming for site-scoped values is
- `LP_<SITE>_<FIELD>` (e.g. `LP_HN_USERNAME`, `LP_GH_TOKEN`); the
- unprefixed `LP_USERNAME` / `LP_PASSWORD` form is the generic fallback.
-- The `getEnv` tool only reads variables whose name starts with `LP_`.
- Everything else (provider API keys, system env, third-party secrets)
- reports "not set" so the model can't probe for it. `getEnv` returns the
- *value* to the model: fine for non-secret config like base URLs, never
- call it on credentials (use `$LP_*` placeholders in fill values instead).
-- `--obey-robots`, `--http-proxy`, `--user-agent`, and the rest of the
- browser-level CLI flags apply to `agent` the same way they apply to
- `serve`, `fetch`, and `mcp`.
-- REPL prompts are persisted to `.lp-history` in plaintext (no encryption).
- Anything you type at the prompt, including natural-language context that
- accompanies a `/login`, lands in that file. Delete it or move out of
- sensitive directories if you don't want it retained.
-- `save` rejects empty, absolute, and `..` paths, but does **not** follow
- up on symlinks. On a shared filesystem, a pre-existing symlink at the
- target would be written through to whatever it points at. Prefer a fresh
- directory you own when saving in untrusted environments.