Skip to content

Lightpanda: the headless browser designed for AI and automation

Update agent-tutorial.md

katie-lpd07727186da27
diff --git a/docs/agent-tutorial.md b/docs/agent-tutorial.md
index a7e3dad41..d9e204721 100644
--- a/docs/agent-tutorial.md
+++ b/docs/agent-tutorial.md
@@ -1,278 +1,119 @@
-# Agent tutorial — Hacker News, end-to-end
+# Agent tutorial, Hacker News end-to-end
-This walks you from "I just built `./lightpanda`" to a recorded,
-replayable JavaScript browser script, then captures the same flow from
-an external MCP client. Every section ends with a command you can run;
-nothing references later sections.
+This tutorial will walk you through how to use Lightpanda Agent to create a
+reproducible JavaScript browser script.
-For the flag/command/tool tables, see [agent.md](agent.md). This
-document is the tutorial; that one is the reference. For the JavaScript
-runtime contract, see [agent-script.md](agent-script.md).
-
-## What you'll build
-
-One session against Hacker News:
-
-1. Log in with your account.
-2. Confirm the login by reading the username out of the header.
-3. Save the whole flow to a `.js` file.
-4. Run it offline, with no LLM.
-5. Add local JavaScript logic around `extract(...)` results.
-6. Save the same flow as a script from an external agent over MCP.
+For flag and command tables, see [agent.md](agent.md).
## Prerequisites
-- `./lightpanda` on your PATH (build with `zig build`).
+- `./lightpanda` on your PATH.
- A Hacker News account.
-- One LLM API key for sections that use natural language — Anthropic,
- OpenAI, Gemini, or a local Ollama. Recorded `.js` scripts run with no
- key at all.
-
-Export your HN credentials as `LP_*` env vars. The convention is
-`LP_<SITE>_<FIELD>` — a short site identifier (`HN` for Hacker News,
-`GH` for GitHub, …) lets you keep credentials for multiple sites in
-your environment without collisions. The unprefixed `LP_USERNAME` /
-`LP_PASSWORD` form is the generic fallback when you only have one
-site.
+- An LLM API key for the natural-language sections (Anthropic, OpenAI,
+ Gemini, or a local Ollama). Recorded `.js` scripts need no key.
-In **bash** or **zsh**:
+Export your HN credentials as `LP_*` env vars. The `LP_` prefix
+matters for security: Lightpanda only resolves these placeholders
+inside its own subprocess (so your password never reaches the LLM),
+and the `getEnv` tool refuses any variable that doesn't start with
+`LP_` (so the LLM can't read your other secrets).
```console
export LP_HN_USERNAME="your-hn-handle"
export LP_HN_PASSWORD="your-hn-password"
```
-In **fish**:
-
-```fish
-set -gx LP_HN_USERNAME "your-hn-handle"
-set -gx LP_HN_PASSWORD "your-hn-password"
-```
-
-The `LP_` prefix matters. The agent resolves `$LP_*` references
-*inside* the Lightpanda subprocess, so the literal secret never enters
-the LLM context; and the `getEnv` tool refuses to read anything that
-doesn't start with `LP_`, so the model can't probe your other env
-vars.
-
-Verify they're set before continuing — substitution fails silently if
-a variable is missing (the literal `$LP_HN_USERNAME` ends up typed
-into the form), and the `/fill` confirmation message intentionally
-echoes the placeholder name rather than the resolved value, so the
-response text won't tell you. Confirm directly:
+Check the variables are set. If one is missing, `/fill` will silently
+type the literal `$LP_HN_USERNAME` into the form rather than your
+username:
```console
./lightpanda agent --no-llm
> /getEnv LP_HN_USERNAME
```
-`/getEnv` returns the literal value if set, or "not set" if missing.
-Only `$LP_*` references in fill values are substituted; other `$`
-characters in your password (`my$ecret`, `$5.99`) are passed through
-verbatim.
-
-## 1. First contact: the REPL
+## 1. Start the REPL
```console
./lightpanda agent
```
-On startup the agent prints a `Lightpanda Agent (version)` banner. The
-status bar under the prompt then shows the model it resolved (a detected
-cloud key, or a local Ollama server if that's all it finds), and the help
-line tells you whether natural language is available or you're in the
-basic `--no-llm` REPL. The REPL writes its history to `.lp-history` in the
-working directory, so up-arrow works across runs.
-
-Try the meta commands:
+The status bar under the prompt shows the resolved model and whether
+natural language is available. REPL history lives in `.lp-history` in
+the working directory.
```
-> /help
-> /help goto
+> /help # list every browser tool
+> /help goto # JSON schema for one tool
> /quit
```
-`/help` lists every browser tool. `/help <tool>` prints its JSON
-schema. `/quit` exits cleanly. If you have no API key yet and want to
-poke around without an LLM, `./lightpanda agent --no-llm` forces the
-basic REPL.
+No API key? `./lightpanda agent --no-llm` runs the slash-commands-only
+REPL.
-## 2. The shortest possible win: `--task`
+## 2. Run a single task
-Before doing anything complicated, prove the LLM + browser stack
-works end-to-end:
+Before doing anything complicated, run a one-shot task to check
+everything works:
```console
./lightpanda agent --task "what is the top story on news.ycombinator.com?"
```
-`--task` runs a single user turn, prints the final answer on stdout,
-and exits. Tool calls, progress, and errors all go to stderr, so
-redirecting stdout gives you a clean answer:
+`--task` runs one user turn, prints the answer on stdout, exits.
+Tool calls and progress go to stderr, so redirecting gives you a
+clean answer:
```console
./lightpanda agent --task "top story on news.ycombinator.com?" > out.txt
```
-If you need to feed the model a local file, repeat
-`-a <path>` (or `--attach <path>`) for each one.
+Use `-a <path>` (repeatable) to attach local files.
-## 3. Driving the browser by hand
+## 3. Log in to Hacker News
-Now back to the REPL. We'll write the HN login flow one command at a
-time so you can see how each step depends on what the previous one
-showed.
+Paste these into the REPL in order:
```
> /goto https://news.ycombinator.com/login
-```
-
-`/goto` takes a single URL argument (positional, optionally quoted). The page
-is now loaded.
-
-> The REPL scripting surface is slash commands. `click '#foo'` (no leading slash) is
-> forwarded to the LLM as a natural-language prompt; only `/click '#foo'`
-> runs as a command. TAB completion in the REPL helps you find the right
-> tool name.
-
-Inspect it before clicking anything:
-
-```
-> /tree
-```
-
-`/tree` prints the semantic tree to stdout. Two forms are visible —
-the login form and the create-account form below it — and each one
-contains two unlabeled textboxes:
-
-```
-8 form
- 13 'username:'
- 15 [i] textbox
- 18 'password:'
- 20 [i]
- 22 [i] button 'login' value='login'
-30 form
- 35 'username:'
- 37 [i] textbox
- …
-```
-
-Notice the textboxes have no accessible name — "username:" is a
-sibling text node, not a `<label for="…">`. This is typical of older
-pages. The ARIA-name tool reflects that:
-
-```
-> /findElement role=textbox name=username
-[]
-```
-
-Empty result. Slash commands accept a single positional argument
-(for tools with one required field), `key=value` pairs, or a raw
-`{json}` blob; `findElement` filters by accessible name, which we
-don't have here.
-
-For unlabeled login forms, jump to `detectForms`, which reads the
-HTML directly and surfaces each form's `action` plus each input's
-`name` attribute:
-
-```
-> /detectForms
-```
-
-You'll see two forms, the first with `action: "login"` and fields
-named `acct` and `pw`. That's enough to synthesize the CSS selector
-yourself: scope by form action to avoid colliding with the
-create-account form, then key on the input's `name` attribute.
-
-**Selector rule, load-bearing:** the click-family tools (`/click`,
-`/fill`, `/hover`, `/selectOption`, `/setChecked`) accept CSS selectors
-only. The backend node IDs `findElement` and `detectForms` return are
-invalidated by any DOM mutation, and they cannot be serialized into
-durable JavaScript recordings — a session that uses them is not
-replayable. Always
-synthesize a CSS selector from the attributes (`id`, `class`,
-`name`, `action`, `tag_name`) and use that.
-
-You don't have to hand-roll the selector. Every node in the `/tree`
-output carries a `backendNodeId` (the leading number on each line), and
-`/nodeDetails` turns one into a durable selector for you:
-
-```
-> /nodeDetails backendNodeId=15
-```
-
-It returns a ready-to-use CSS `selector` that resolves to that node
-(plus its tag, id, class, name, and attributes) — the canonical way to
-go from a `/tree` (or `/findElement`) hit to a selector you can paste
-into `/click` or `/fill`. We reached for `/detectForms` above because
-the two login/signup forms share field names, so a form-scoped selector
-(keyed on `action`) is cleaner here — but `/nodeDetails` is the quickest
-path whenever you have a single `backendNodeId` and just want its
-selector without guessing.
-
-Now fill the form:
-
-```
> /fill selector='form[action="login"] input[name="acct"]' value='$LP_HN_USERNAME'
> /fill selector='form[action="login"] input[name="pw"]' value='$LP_HN_PASSWORD'
> /click selector='form[action="login"] input[type="submit"][value="login"]'
> /waitForSelector '#logout'
```
-`$LP_*` references in `/fill` values are resolved at execution time
-inside the subprocess. The LLM never sees the literal credential.
-
-The `/waitForSelector '#logout'` line is doing two jobs at once, and it's
-worth unpacking because the pattern recurs in every recorded script:
-
-- **It's a synchronization point.** `/click` on the submit button
- returns as soon as the click dispatches, not after the server
- responds. Without a wait, the next command races the login redirect
- and may run against the *pre-login* DOM. `/waitForSelector '<sel>'`
- blocks until that selector appears, so the script resumes only after
- HN's logged-in page has rendered.
-- **It's an implicit assertion.** HN renders the `#logout` link only
- when the session is authenticated. If the credentials are wrong (or
- HN throws up a captcha, or rate-limits, or the form layout
- changed), `#logout` never appears and `/waitForSelector` times out —
- the script fails loudly at the line where the failure actually
- happened, instead of silently succeeding and producing garbage
- downstream.
-
-You generally want a `/waitForSelector` like this after every state-changing
-action that triggers async work: pick a selector that *only* exists
-in the post-action state, and you get free regression protection.
-Waiting on the URL (`location.pathname === "/news"`) or a generic
-element that exists on both pages is weaker — both can be true before
-the navigation finishes.
-
-Confirm by pulling structured data off the page. `/extract` takes a
-JSON schema object — each value describes what to lift out, and the
-whole result is printed to stdout as one JSON object. The simplest
-form is a flat selector lookup:
+A few things worth knowing:
+
+- **Slash commands only.** `click '#foo'` is forwarded to the LLM;
+ only `/click '#foo'` runs as a command. TAB completes tool names.
+- **`/waitForSelector '#logout'` is both a sync point and an
+ assertion.** It blocks until HN's logged-in DOM renders. If the
+ credentials are wrong (or the website throws a captcha, or the layout
+ changes), the command times out at the line where the failure
+ actually happened. Use this pattern after every state-changing
+ action.
+- **Selectors are CSS only.** The click-family tools (`/click`,
+ `/fill`, `/hover`, `/selectOption`, `/setChecked`) accept CSS
+ only. Backend node IDs are invalidated by any DOM mutation and
+ can't be serialized into recordings.
+
+Confirm the login worked:
```
> /extract '{"karma": "#karma"}'
{"karma":"42"}
```
-The schema grammar is small but covers the cases you'd reach for:
+`/extract` takes a JSON schema and prints one JSON object to stdout.
+Schema grammar:
-- `"<sel>"` — `textContent.trim()` of the first match (string, or
- `null` if no match).
-- `""` — the matched element's own text (only meaningful inside a
- `fields` block, where there's an outer element to refer to).
-- `["<sel>"]` — text of every match (string array).
-- `{"selector": "<sel>", "attr": "<name>"}` — the first match's
- attribute value.
-- `[{"selector": "<sel>", "fields": {…}}]` — array of objects, where
- each `fields` entry is resolved relative to the matched element.
+- `"<sel>"`: text of the first match.
+- `["<sel>"]`: text of every match.
+- `{"selector": "<sel>", "attr": "<name>"}`: attribute of the first match.
+- `[{"selector": "<sel>", "fields": {…}}]`: array of records, with
+ each `fields` entry resolved relative to the matched element.
-After a page-changing action (click, navigation, form submit) the
-previous `/tree` snapshot is stale; re-inspect before the next
-interaction. Hop back to the front page and pull the story list to
-exercise the structured form:
+Now go to the front page and pull the story list:
```
> /goto https://news.ycombinator.com
@@ -290,42 +131,38 @@ exercise the structured form:
'''
```
-Triple-quoted (`'''` or `"""`) values let a schema span multiple lines
-— the REPL keeps reading until it sees the matching closing quote.
-The result is a single JSON object printed to stdout:
-`{"topStories":[{"rank":"1","title":"…","url":"…"}, …]}`.
+Triple-quoted values let a schema span multiple lines.
-The schema is parsed in Zig before the page-side walker runs, so a
-typo like a stray comma surfaces here as a plain `Error: InvalidParams`
-instead of a confusing V8 stack trace.
+### How we got those selectors
-## 4. Saving the session
+Skip this if you're happy treating them as given.
-The same flow, but exported to a file. In the same REPL, retype the
-sequence — login (`/goto`, two `/fill`s, `/click`, `/waitForSelector`),
-then the front-page hop and structured pull (`/goto`, multi-line
-`/extract`) — then save it:
+`/tree` prints the semantic tree. On the login page you'll see two
+forms (login and signup) with unlabeled textboxes, which means
+`/findElement role=textbox name=username` returns nothing.
-```
-> /save hn_login.js
-```
+`/detectForms` reads the HTML directly and surfaces each form's
+`action` plus each input's `name`. The first form has `action: "login"`
+and fields named `acct` and `pw`, which gives us the form-scoped
+selector (`form[action="login"] input[name="acct"]`) that won't
+collide with signup.
-In the basic REPL (`--no-llm`) `/save` transcribes the session
-deterministically; with an LLM it synthesizes an equivalent idiomatic
-script. Either way `/quit` when you're done.
+`/nodeDetails backendNodeId=<n>` is the alternative: it returns a
+ready-to-use CSS selector for any node ID from `/tree`.
-Inspect the result:
+## 4. Save the session as a script
-```console
-cat hn_login.js
+Retype the login plus front-page sequence in a single REPL session,
+then:
+
+```
+> /save hn_login.js
+> /quit
```
-You should see the seven mutating commands and nothing else — no
-`/tree`, no `/markdown`, no read-only lookups. The `--no-llm` recorder
-keeps only state-mutating tools (each tool's `isRecorded` flag), so
-read-only inspection never pollutes the script; `/extract` *is* recorded
-(it changes what the script can read on replay even though it doesn't
-mutate the page). The saved file is JavaScript:
+In `--no-llm` mode, `/save` transcribes the session deterministically.
+With an LLM, it synthesizes an idiomatic script. The result is
+JavaScript:
```js
goto("https://news.ycombinator.com/login");
@@ -337,36 +174,29 @@ goto("https://news.ycombinator.com");
extract({ topStories: [{ selector: ".athing", fields: { rank: ".rank", title: ".titleline > a", url: { selector: ".titleline > a", attr: "href" } } }] });
```
-Natural-language REPL turns are never saved as executable JavaScript.
-In the deterministic `--no-llm` transcription, the prompt that produced
-a set of recorded actions is kept as a `//` comment above them, so the
-script stays readable. The LLM `/save` path is different: it rewrites
-the whole session into an idiomatic script and is told to emit
-JavaScript only, so it generally drops those comments rather than
-preserving them verbatim.
+Only state-mutating commands are recorded; read-only ones (`/tree`,
+`/markdown`) are dropped. `/extract` is recorded because it shapes
+what the script returns.
-## 5. Running deterministically
+## 5. Replay the script without an LLM
```console
./lightpanda agent hn_login.js
```
-No `--provider`, no LLM, no token spend. The saved script runs top
-to bottom against a fresh browser. This is the form you want for
-regression tests and CI. From inside the REPL, `/load hn_login.js`
-runs the same script against the current session.
-
-The script's completion value — its last top-level expression — is
-printed automatically (objects and arrays as JSON). Because the saved
-script ends with the `extract(...)` call, you already get clean JSON on
-stdout with nothing to edit:
+No `--provider`, no API key, no token spend. The script's last
+expression is printed automatically as JSON. Because the saved script
+ends with `extract(...)`, you get clean JSON on stdout:
```console
./lightpanda agent hn_login.js > stories.json
```
-If you want to reshape the output first, assign the result and end with
-a bare expression — that final value is what prints:
+From inside the REPL, `/load hn_login.js` runs the same script against
+the current session.
+
+To reshape the output, assign the result and end with a bare expression
+(the final value is what prints):
```js
const topStories = extract({
@@ -380,24 +210,17 @@ const topStories = extract({
}]
});
-topStories; // printed automatically as JSON
+topStories;
```
-`/login`, `/logout`, and `/acceptCookies` are REPL-only LLM triggers. A
-script saved with `/save` never contains them; `/save` captures the
-resulting browser tool calls instead. Lines that are neither slash commands nor
-comments are also REPL-only conveniences, not script syntax.
+## 6. Add your own JavaScript logic
-## 6. Local JavaScript logic
+Agent scripts run in a separate JavaScript context from the page. No
+`window`, `document`, DOM API, `require`, or `process`. Browser
+interaction happens through the installed primitives.
-Agent scripts run in a separate JavaScript context from the web page.
-There is no `window`, `document`, DOM API, `require`, `process`, or
-Node standard library in that context. Browser interaction happens
-through the installed primitives, and `extract(...)` is the usual way
-to move page data into local script logic.
-
-For example, turn the extraction result into a smaller report without
-running any page-side JavaScript:
+Use `extract(...)` to move page data into local logic, then process it
+with normal JavaScript:
```js
goto("https://news.ycombinator.com");
@@ -414,29 +237,18 @@ const topStories = extract({
}]
});
-const report = topStories.map((story) => ({
- rank: story.rank,
- title: story.title,
- url: story.url
-}));
-
-report; // printed automatically as JSON
+topStories.map((s) => ({ rank: s.rank, title: s.title, url: s.url }));
```
-Use the `evaluate(...)` primitive only when you intentionally want to run a
-string in the current page's JavaScript context. Page evaluate cannot see
-agent variables or call `goto`, `extract`, and the other agent
-primitives.
+Use `evaluate(...)` only when you intentionally want a string to run in
+the page's JavaScript context. Page evaluate cannot see agent
+variables or call agent primitives.
-## 7. Same flow, external agent (MCP)
+## 7. Use Lightpanda from another agent (MCP)
-Everything above used Lightpanda's built-in agent. If you're driving
-Lightpanda from a different agent (Claude Code, a custom MCP client,
-your own harness), use `lightpanda mcp` instead — same browser tools,
-no API key on the Lightpanda side, the calling agent supplies the
-LLM.
-
-Register the server with your MCP client:
+If you're driving Lightpanda from a different agent (Claude Code, a
+custom MCP client, your own harness), use `lightpanda mcp` instead.
+The calling agent supplies the LLM, so Lightpanda needs no API key.
```json
{
@@ -449,38 +261,32 @@ Register the server with your MCP client:
}
```
-### Save a script over MCP
-
-The MCP server has no LLM of its own — your external agent is the brain.
-Drive the browser with the usual tools, then hand back a script with the
-`save` tool:
+Drive the browser with the usual tools (`goto`, `fill`, `click`,
+`waitForSelector`), then hand back a script with the `save` tool:
-1. Run the browser tools you'd call anyway: `goto`, `fill`, `click`,
- `waitForSelector`. The server resolves `$LP_*` placeholders inside the
- subprocess, so credentials never reach your agent's context.
-2. When the task is done, synthesize a `.js` script from the steps that
- mattered — call the builtins as JavaScript functions with the same
- object arguments — and call `save { "path": "hn_login.js", "script":
- "goto(\"...\");\n..." }`. The path must be relative and free of `..`;
- the response reports the absolute location and line count.
+```json
+{
+ "tool": "save",
+ "args": {
+ "path": "hn_login.js",
+ "script": "goto(\"...\");\n..."
+ }
+}
+```
-The `save` tool's description carries the same guidance the REPL's
-`/save` gives its LLM (prefer builtins, drop dead-ends, keep `$LP_*`
-placeholders), and any literal `LP_*` value is scrubbed back to its
-placeholder before the file is written. The output uses the same
-JavaScript format as `/save hn_login.js` from section 4 and runs
-unmodified:
+The path must be relative and free of `..`. Literal `LP_*` values are
+scrubbed back to placeholders before the file is written. The output
+runs unmodified:
```console
./lightpanda agent hn_login.js
```
-## Where to go next
+## What next?
-- [agent.md](agent.md) — full reference: every flag, every slash
- command, every browser tool, plus the security model and
- auto-detection rules.
-- [agent-script.md](agent-script.md) — JavaScript runtime, primitives,
- return values, and complete script examples.
-- `lightpanda mcp --help` and `lightpanda agent --help` — current
- flag listings straight from the binary.
+- [agent.md](agent.md): full reference for every flag, slash command,
+ and browser tool.
+- [agent-script.md](agent-script.md): JavaScript runtime, primitives,
+ return values.
+- `lightpanda mcp --help` and `lightpanda agent --help`: current
+ flags straight from the binary.