Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

feat(jev-browser): a real browser that reads real pages into a spreadsheet (#133)

Dee4ab2946147b7
diff --git a/store/agents/jev-browser/AGENTS.md b/store/agents/jev-browser/AGENTS.md
index 00c88b226..7f6c8d1d6 100644
--- a/store/agents/jev-browser/AGENTS.md
+++ b/store/agents/jev-browser/AGENTS.md
@@ -1,75 +1,170 @@
# Jev Browser in OpenHarness
-On the left is a small made-up travel site inside a browser window, and **Jev operates it by
-itself**. Jev is TypeSafe's System One model. It never sees the screen. Each step the page is written
-out as text and Jev picks ONE element out of the list, a new list every step. A booking takes a few
-seconds. Every booking is checked against its task.
+**This tool turns a person who cannot write a scraper into someone who can read the web into a
+spreadsheet.** They point it at a page. It opens a real Chrome, walks the real site, and for every
+thing on it writes a row: title, price, date, whoever, whatever they asked for. They leave with
+`results.csv` and a job file they can run again next month.
-On the right, you edit `site.json`. This is the ONLY file you edit. The viewer watches it and starts
-again from the first task the moment you save.
+**Jev never writes a value.** For each field it is shown the numbered pieces of text that are really
+on the page, and it picks one. So a cell holds the page's own words, or nothing. Nothing is invented.
-## The site file
+On the left is the pane: the live browser, Jev's yes-or-no on every link of the page it is on, and
+the rows as they land. On the right, you. You edit `browse.json`. That one file is the job and the
+recipe. The viewer watches it and reloads on every save.
+
+`$JEV_DSH` below is the harness's own folder. The workspace sets it for you; `echo $JEV_DSH` shows
+it. If it is empty, you are not in a Jev Browser workspace: say so rather than guessing a path.
+
+## Your job, in this order
+
+1. **Find out what they want, and from where.** A start address, what one "thing" is, and the
+ columns they want. If they paste a page, open it yourself first (see "Look before you write the
+ job") so the field asks match the words really on it.
+2. **Write `browse.json`.** Validate with `node "$JEV_DSH/toolchain/check.mjs"`.
+3. **Ask them to press Start.** The pane owns the browser and you do not have its address, so you
+ cannot press it. Then watch `.harness/verdict.json` (see "While it runs").
+4. **Read `results.csv`.** Its columns are `item`, `page title`, `address`, then each of your
+ fields with a confidence column beside it. The address is the page each row came off, so every
+ number can be checked: say that when you report. Derive whatever you like with a script (a sort
+ key out of a sentence, a total, a ranking). Never edit the file, and never retype a value from a
+ page into the chat as if it had been collected.
+5. **Fix what came back thin.** A field found on few pages is usually asked in the wrong words, or
+ it is on the list page and not on the thing's own page. Reword, save, run again.
+6. **Tell them what they have**, in plain words: how many things, what is missing, what to do next.
+ Offer the file. If they want the same thing next month, tell them the job file is the recipe.
+
+## The job file
```jsonc
{
- "title": "Jev Browser",
- "description": "One line about this run.",
- "site": "SkyHop", // the made-up site's name, shown in the tab and the URL
- "stepMs": 170, // one Jev decision per step. 40 to 2000.
- "distraction": 0.25, // 0 to 1. Cookie walls, pop-ups, decoys, layout shifts. THE DIAL.
- "flights": 7, // result rows per search. 4 to 9.
- "maxSteps": 45, // a task fails if it takes more steps. 12 to 200.
- "seed": 11,
- "style": "How Jev should work, in plain words. It is the first line of the text Jev reads.",
- "tasks": [ // 1 to 40 bookings, run in order, then again with new flights
- { "from": "SFO", "to": "JFK", "day": "Friday", "pick": "cheapest", "nonstop": true,
- "name": "Ada Park", "email": "[email protected]", "bags": 1 }
- ]
+ "task": "Every flat for rent in the search results, with rent and address",
+ "start": "https://example.com/search?area=leeds", // the page to begin on. "demo" is the made-up job board this harness serves itself
+ "item": "a flat for rent", // one of the things. Used in every question, so make it concrete
+ "fields": [
+ { "id": "address", "name": "Address", "ask": "the street address" },
+ { "id": "rent", "name": "Rent", "ask": "the monthly rent" },
+ { "id": "beds", "name": "Bedrooms", "ask": "how many bedrooms" },
+ { "id": "garden", "name": "Garden?", "ask": "Does it have a garden?", "type": "yesno" },
+ { "id": "state", "name": "Condition", "ask": "what condition it is in", "type": "score",
+ "levels": ["needs work", "liveable", "newly done"] }
+ ],
+ "keep": "only flats that allow pets", // optional. A yes/no on every thing, written to the file
+ "maxItems": 60, // how many things to collect
+ "maxPages": 25, // how far down the list to walk (page 2, page 3…)
+ "sameSiteOnly": true, // stay on the site the start address is on
+ "alsoVisit": [], // other hosts it may reach, if the things live elsewhere
+ "show": true // a window the person can watch and take over
}
```
-`pick` is `cheapest`, `earliest` or `latest`. `bags` is 0 to 3. Airport codes are up to 4 letters.
-Names and emails are made up. Use `example.com` addresses.
+- **`fields`**: up to 12. `pick` (the default) takes the exact text off the page. `yesno` is Jev's
+ judgement about the thing. `score` puts it on your named scale. Every field gets a confidence
+ column in the spreadsheet.
+- **`item`** goes into the question asked about every link, so "a flat for rent" works and "an item"
+ does not.
+
+## How it walks a site
+
+One Jev call per page, and that call holds everything worth asking about it.
+
+- **A list page**: what kind of page it is, plus **one yes-or-no for every link on it** ("is
+ *Senior Python Engineer* the title of a job posting, rather than part of the site's menu?"), plus
+ which link is the next page. A page with 120 links is 122 questions in one call, because many
+ questions about one page cost about the same as one.
+- **One thing's page**: what kind of page it is, plus **every field at once**, each a choice over
+ the page's own numbered text. Six fields cost one call.
+
+It opens the things it found, then asks for the next list page, until `maxItems` or `maxPages`.
+
+## Look before you write the job
+
+Open the page yourself and see what the reader sees, so your asks match the page's words:
+
+```sh
+node --input-type=module -e "
+import { openChrome } from '$JEV_DSH/toolchain/chrome.mjs'
+import { readPage } from '$JEV_DSH/viewer/page.mjs'
+const c = await openChrome({ profileDir: '/tmp/jev-look', show: false, allowedHosts: ['example.com'] })
+await c.go('https://example.com/search')
+const p = await readPage(c)
+console.log(p.title); for (const b of p.blocks.slice(0, 30)) console.log(' block', JSON.stringify(b.text))
+for (const l of p.links.slice(0, 30)) console.log(' link ', l.label, '->', l.path)
+await c.close()"
+```
+
+Reading that list:
+
+- **Each distinct piece of text is offered once.** A value that also appears higher up the page is
+ listed at its first appearance, not twice. Search the whole list before you tell anyone a value
+ is missing.
+- **A label and its value are one block**, as `Availability: In stock (19 available)`, with the
+ value on its own as the first part.
+- **`parts` is what a column can take instead of the whole line.** A line splits on ` · `, ` | `,
+ a dash or a bullet, and any number inside it is offered on its own: `In stock (19 available)` has
+ the part `19`, so "how many are in stock" can come back as a number you can sort.
+- If a value really is not in that list, Jev cannot return it: it only picks. Say so, and ask for
+ something that is on the page. Text drawn as a picture, loaded after a click, or shown on hover
+ is not on the page as far as the reader is concerned.
+
+## While it runs
+
+`.harness/verdict.json` is the truth. Read it; do not edit it.
+
+```jsonc
+{ "ready": false,
+ "summary": "24 collected from 26 pages, 34 links judged, 26 calls, $0.0023",
+ "run": { "client": "openrouter", // "mock" means no key: see "Without a key"
+ "rows": 24, "pages": 26, "linksJudged": 34, "calls": 26, "costUsd": 0.0023,
+ "fields": [ { "name": "Salary", "found": 24, "of": 24, "avgConfidence": 1, "thin": false } ] },
+ "findings": [ { "severity": "warning", "kind": "field", "message": "…" } ] }
+```
+
+- `run.rows` climbing means it is working. Expect about two seconds a thing: the page load, not Jev.
+- `fields[].thin` is the one to act on: it means the column came back on under 60% of the things.
+- A run that ends with no next-page link is a clean finish, not a failure: `ready` goes true.
+- Trouble on a page is a finding, and the run carries on.
+
+## What it will not do, however it is asked
+
+The refusals live in `toolchain/chrome.mjs`, next to the only code that touches the page.
+
+- **It only reads.** It never submits a form and never presses anything that reads like pay, buy,
+ checkout, delete, send, apply or book.
+- **It never types a password, card number or one-time code.** If a site needs a login, the person
+ signs in themselves in the window; the profile is kept in the workspace, so next time it is
+ already signed in.
+- **It stays on the sites the job names**, and only on http and https. Downloads are refused.
-## Your job
+Never tell a person you can work around these, and never ask them for a password.
-Write bookings that are fun to watch, then find out where Jev breaks.
+## Being straight about it
-- **Write the tasks.** Mix the three `pick` rules, nonstop and not, and different bag counts, so
- the right flight is a different row each time and the passenger page needs different clicks.
-- **Set the dial.** At `distraction` 0 every booking should come out exactly right in about ten
- steps. Raise it and clicks start landing in the wrong place because the page moved between
- reading and clicking. The person can also move the slider in the pane.
-- **Set the pace.** `stepMs` near 170 looks like a fast agent. 800 or more lets a person read each
- step.
-- **Write the style line.** With a real key, Jev reads it on every step.
-- **Report what you see.** Read `.harness/verdict.json`: bookings done, the share that were exactly
- right, steps each, and clicks lost to layout shifts. It also lists the most recent wrong bookings
- and why they were wrong. Say at which distraction the share that is right drops below 9 in 10.
+- A site may say in its terms that it does not want to be read this way, and some sites charge for
+ an API that gives the same data. Say so once, and let the person decide. Do not go around a
+ block, a login wall, a rate limit or a robots rule.
+- Take what is asked for and no more. `maxItems` is a page load each, so it costs the site more
+ than it costs you: set it to what the person actually needs, not to the maximum.
+- The rows are what the page said on the day it was read. If that matters, say when it was read.
+- `results.csv` holds the person's data. Do not copy it anywhere.
-Do NOT just ship the template. Every run should have its own tasks and a dial you chose on purpose.
+## Without a key
-Validate with `node "$JEV_DSH/toolchain/check.mjs"`.
+`"client": "mock"` in the verdict means there is no Jev key, and an offline stand-in is answering by
+word-matching. It shows the plumbing; its rows are not worth acting on. Tell them to paste a key
+into the **Jev · live mind** panel in the pane (an OpenRouter key from `openrouter.ai/keys` takes
+about a minute). Never ask them to paste a key into the chat.
## Rules
-- Edit only `site.json`. Keep it valid JSON. A bad edit does not crash the demo. The pane shows the
- error and keeps running on the last good settings.
-- Never edit `.harness/verdict.json`. The viewer writes it.
-- The viewer is already running in the left pane. Never propose opening a browser, changing ports
- or running a second server. The "browser" in this harness is a drawing inside the pane. It never
- loads a real website.
-- You can ask Jev yourself through `toolchain/jev.mjs` (`evaluate`, and the `jev.choice`,
- `jev.noul`, `jev.score` builders), for example to see which element it picks on one page.
-- Without `TYPESAFE_API_KEY` the harness runs on a deterministic offline stand-in, and the pane
- says `MOCK`. It is there so the demo runs anywhere. It is not Jev's judgement, so do not describe
- its results as Jev's.
-- This is a demo. The site, flights, prices and people are made up. Nothing is really booked. Do not
- present the times as a benchmark of any product.
+- Keep `browse.json` valid JSON. A bad edit keeps the last good job and shows the error.
+- Do not edit `.harness/verdict.json` or `results.csv`. The viewer writes them.
+- Never propose opening a browser yourself, changing ports, or running a second server. The pane
+ owns the browser, and Start, Stop and the address bar are the person's buttons, not yours. The
+ one browser you may open is a read-only look with `openChrome` (below), which touches nothing.
+- Chrome must be on the machine. `toolchain/doctor.sh` says whether it is.
## Definition of done
-- `site.json` parses and passes `toolchain/check.mjs`.
-- The pane runs your tasks without an error banner.
-- You told the person what dial you chose, how many bookings come out exactly right, how many steps
- they take, and where it starts to fail.
+- `browse.json` passes `toolchain/check.mjs` and names a real start address.
+- The verdict shows a live `client`, rows collected, and no field marked `thin`.
+- You read `results.csv` and told the person what is in it, what is missing, and what it cost.
diff --git a/store/agents/jev-browser/README.md b/store/agents/jev-browser/README.md
index 20a5d9e15..4efd9e32c 100644
--- a/store/agents/jev-browser/README.md
+++ b/store/agents/jev-browser/README.md
@@ -1,104 +1,138 @@
# Jev Browser
-**Jev operates a web browser, live, and books a flight in a few seconds.** Jev is TypeSafe's System
-One model. It never sees the screen and never writes a word. At every step the page of a small
-made-up travel site is written out as text: the task, the page, and every element that can be
-acted on. Jev answers in **one call**:
-
-| question | type | what it drives |
-|---|---|---|
-| `action` | choice, a new option list every step | the ONE element to act on next |
-| `done` | noul (yes/no) | is the task already complete |
-| `obstacle` | noul (yes/no) | is something covering the page |
-
-Clicking an input fills it with the right value from the task. In the fast browser agents people
-built with Jev, a small text model wakes up only to type. Here a plain lookup plays that part.
-
-The pane shows the site in a browser window with a cursor that flies to each chosen element, rings
-on the page showing the probability Jev gave the top candidates, the exact text Jev reads, a timer,
-and a live panel with decisions per second, latency, tokens and cost. Every booking is checked
-against the task: the right flight, the right passenger, the right bags, no extras.
-
-This is a harness for OpenHarness. The agent on the right edits `site.json`. The viewer on the left
-runs the site and asks Jev for every step.
-
-## Play with it
-
-- **Click anything on the page yourself.** Your click is a real step. Jev has to deal with what
- you did.
-- **Throw a pop-up** at it mid-task.
-- **Distraction** and **Step** sliders change the dial and the pace live.
-- **Show Jev's mind** toggles the probability rings. **Pause** then **Step** holds each page for a
- moment so you can read the rings before the cursor moves.
-- Next task, Pause, Step, Reset.
-
-## The honest dial
-
-`distraction` (0 to 1) adds cookie walls, pop-ups, decoy buttons, upsells and **layout shifts**. A
-layout shift lands between reading the page and clicking it: a pop-up opens, or a sponsored row
-pushes the results down, and the click goes to the old spot and hits whatever is there now. That is
-how real web pages break real agents. Nothing else is random.
-
-Measured with the offline stand-in over 300 bookings at each level:
-
-| `distraction` | bookings exactly right | steps per booking | clicks lost per booking |
-|---|---|---|---|
-| 0.00 | 100% | 10.0 | 0.0 |
-| 0.25 (default) | 94% | 11.5 | 0.7 |
-| 0.50 | 92% | 13.2 | 1.5 |
-| 0.75 | 85% | 14.9 | 2.2 |
-| 1.00 | 81% | 17.2 | 3.3 |
+**Point it at a web page. Get a spreadsheet.** Say what one "thing" is on that page and which
+columns you want, and a real Chrome walks the site: it opens every thing, pulls out your columns,
+and writes `results.csv` while you watch. No code, no selectors, no copy and paste.
+
+**Every value is text taken off the page.** Jev, TypeSafe's System One model, never writes words. It
+is shown the numbered pieces of text that really are on the page and it picks one. So a cell holds
+the page's own words, or it holds nothing. A made-up price is not possible, because there is nothing
+to make one out of.
+
+This is a harness for OpenHarness. The pane on the left is the browser and the spreadsheet. The
+agent on the right writes the job with you, reads the results, and sharpens what came back thin.
+
+## One call per page
+
+That single call to Jev holds everything worth asking about the page.
+
+- **On a list page**: what kind of page it is, **one yes-or-no for every link on it**, and which
+ link is the next page. 120 links is 122 typed questions, answered together, each with its own
+ probability. In the picture above, the twelve real roles came back at 76 to 88 per cent and the
+ menu links at 2 per cent.
+- **On one thing's page**: every field at once, each a choice over the page's own text.
+
+Many questions about one page cost about the same as one, so this is the cheap way round.
+
+## What it will not do
+
+The refusals sit next to the only code that can touch a page, so nothing above them can go around.
+
+- **It only reads.** It never submits a form, and never presses anything that reads like pay, buy,
+ checkout, delete, send, apply or book.
+- **It never types a password, a card number or a one-time code.** If a site needs a login, you sign
+ in yourself in the window. The profile is kept in your project folder, so the next run is already
+ signed in.
+- **It stays on the sites your job names**, on http and https only, and downloads are refused.
+
+Some sites say in their terms that they do not want to be read this way, and some sell an API for
+the same data. That is your call to make, and the harness will not go around a block or a login.
+
+## Try it in ten seconds
+
+The harness serves its own small job board on your machine, so there is something honest to walk
+before you point it anywhere real. Press **Start**. Only the content of that site is made up: the
+HTTP, the links, the pagination and the browser are all real.
+
+Then change `start` in `browse.json` to a real address.
+
+## The job, which is also the recipe
+
+```jsonc
+{
+ "task": "Every flat for rent in the search results, with rent and address",
+ "start": "https://example.com/search?area=leeds",
+ "item": "a flat for rent",
+ "fields": [
+ { "id": "address", "name": "Address", "ask": "the street address" },
+ { "id": "rent", "name": "Rent", "ask": "the monthly rent" },
+ { "id": "garden", "name": "Garden?", "ask": "Does it have a garden?", "type": "yesno" }
+ ],
+ "maxItems": 60,
+ "maxPages": 25
+}
+```
+
+Run it again next month and you get next month's answer. That file is the whole recipe.
+
+## What you get
+
+- **`results.csv`** in your project folder: one row per thing, every column with the confidence Jev
+ had in it, and the address of the page it came off. Every number in your spreadsheet can be
+ checked against the page that produced it. Download it from the pane or open it from the folder.
+- The agent on the right will count it, tell you what is missing, and sharpen the weak columns.
+
+## Measured
+
+On 2026-09-20 with live Jev (`typesafe/jev-1.13`) through OpenRouter.
+
+| Run | Things | Time | Cost | Result |
+|---|---|---|---|---|
+| The built-in job board, 36 roles | 24 | 33 s | $0.0023 | 144 of 144 cells exactly matched the site's own data, no repeats, nothing skipped |
+| `books.toscrape.com`, a public sandbox | 8 | 21 s | $0.0018 | every title, price, stock count and UPC code right |
+
+The product codes matter: `a22124811bfa8350` is not something a model could write from memory. It
+came off the page, which is the whole point. Small runs are a sanity check, not a benchmark.
+
+## Without a key
+
+The pane runs on an offline stand-in that matches words. It shows the plumbing and it is honest
+about being a stand-in, but its rows are not worth acting on. Paste an OpenRouter or TypeSafe key
+into the **Jev · live mind** panel in the pane. It is saved on your machine in
+`~/.config/typesafe/credentials` and checked with one tiny call.
+
+## The honest limit
+
+The limit is the page, not the model. A site that prints its values as text reads perfectly. A site
+that draws them as pictures, hides them behind a click, or loads them on scroll gives blank cells.
+Those show up as an empty column that the verdict marks thin, not as a wrong answer. The tool would
+rather leave a cell empty than write something the page never said.
## Anatomy
```
jev-browser/
- harness.json # DSH manifest (engine: claude)
- AGENTS.md # tells the agent how to write tasks and what to report
- skills/browser/SKILL.md # the task-writing and verification craft
- template/site.json # the starter tasks and dials
+ harness.json DSH manifest (engine: claude)
+ AGENTS.md the agent's job: write browse.json, read results.csv, sharpen
+ skills/browser/SKILL.md the craft of fields, and what a page can and cannot give
+ template/browse.json the starter job, pointed at the built-in board
toolchain/
- jev.mjs # the Jev client (real TypeSafe API, or an offline stand-in)
- check.mjs # validates site.json
+ chrome.mjs drives a real Chrome over the DevTools protocol, and holds every refusal
+ jev.mjs the Jev client (TypeSafe, Cloudflare or OpenRouter, or the stand-in)
+ check.mjs validates browse.json
viewer.sh setup.sh doctor.sh init-workspace.sh
viewer/
- viewer.mjs # loopback server + the step loop
- sim.mjs # the site: pages, elements with rectangles, layout shifts, scoring
- mock.mjs # the offline stand-in (reads only the page text)
- kit.mjs # loopback-only HTTP/SSE server helpers
- studio.js # the browser window, the cursor, the rings
- jev-hud.js # the "Jev live mind" panel
+ viewer.mjs the server: the job, the browser, the run, results.csv, the verdict
+ page.mjs reads a live page into numbered links and numbered text
+ crawl.mjs the loop and the questions
+ demosite.mjs the made-up job board served on this machine
+ mock.mjs the offline stand-in
+ kit.mjs index.html studio.css studio.js base.css jev-hud.js
test/viewer.test.mjs
```
-## Jev, honestly
-
-With `TYPESAFE_API_KEY` set, every step is a real call to `POST /v1/systemone`. Without a key the
-harness runs on a deterministic local stand-in that reads the same page text and answers the same
-questions, so the demo works offline. The pane badges that mode `MOCK`. The stand-in exercises the
-plumbing. It is not Jev's judgement.
-
-The site, the flights, the prices and the people are made up. Nothing is booked anywhere and no real
-website is touched. The times shown are this demo's pace, not a benchmark of any product.
-
-## Measured with the real model
-
-One short run on 2026-09-20 with live Jev (`typesafe/jev-1.13`) through OpenRouter, about 0.45 s a call once warm. Small samples on made-up data: a sanity check, not a benchmark.
-
-| `distraction` | bookings | exactly right | steps each | clicks lost to layout shifts | cost |
-|---|---|---|---|---|---|
-| 0 | 6 | 6 of 6 | 9.8 | 0 | $0.0017 |
-| 0.6 | 6 | 6 of 6 | 15.0 | 12 | $0.0032 |
+## Requirements
-At distraction 0.6 the real model lost 12 clicks to layout shifts and still got every booking right:
-it noticed the wrong flight on the next page and went back. The offline stand-in scores about 92% there.
+Google Chrome on the machine, and Node 22 or newer. `toolchain/doctor.sh` says whether both are
+there. Set `CHROME_PATH` if Chrome lives somewhere unusual.
## Credit and stewardship
- **Jev** is the work of **TypeSafe AI** (typesafe.ai). This harness is an OpenHarness wrapper that
only calls the public API. It contains no TypeSafe code.
-- The idea of a decision model choosing the next browser action from the page's elements comes from
- the open-source browser agents the community built around Jev. This harness is an independent,
- from-scratch demo on a synthetic site and uses none of their code.
+- The idea of a decision model picking the next browser action out of the page's own elements comes
+ from the open-source browser agents the community built around Jev. This harness is an
+ independent, from-scratch tool and uses none of their code.
+- **Google Chrome** is Google's, and is not included: the harness drives whatever Chrome you have.
- **OpenHarness** (Autonomous) is MIT-licensed. This wrapper is MIT too (see `LICENSE`).
- Built by Autonomous for the OpenHarness store.
diff --git a/store/agents/jev-browser/THIRD_PARTY_NOTICES.md b/store/agents/jev-browser/THIRD_PARTY_NOTICES.md
index c5063e7c5..e65626b05 100644
--- a/store/agents/jev-browser/THIRD_PARTY_NOTICES.md
+++ b/store/agents/jev-browser/THIRD_PARTY_NOTICES.md
@@ -5,6 +5,13 @@ fetch any TypeSafe code. It calls TypeSafe's public API (`POST /v1/systemone`) o
`TYPESAFE_API_KEY` is set. Without one, the harness runs on a deterministic local mock (a stand-in
for the *plumbing*, not for Jev's judgement). Jev and TypeSafe are trademarks of TypeSafe AI.
+**Google Chrome** — Google LLC. This harness drives a copy of Chrome already installed on the
+machine, over the public DevTools protocol. It does not vendor, fetch or modify Chrome, and Chrome
+and Chromium are trademarks of their owners.
+
+**books.toscrape.com** — a public sandbox published by Zyte for practising web scraping. It is used
+once in this harness's README to report a measured run. No code or content from it ships here.
+
**OpenHarness** — Autonomous (this repository), MIT license. This harness is a wrapper that adds a
viewer and agent instructions; it does not modify TypeSafe AI's product.
diff --git a/store/agents/jev-browser/harness.json b/store/agents/jev-browser/harness.json
index e7f8a0c16..fc8520633 100644
--- a/store/agents/jev-browser/harness.json
+++ b/store/agents/jev-browser/harness.json
@@ -2,13 +2,13 @@
"spec": 1,
"id": "autonomous/jev-browser",
"name": "Jev Browser",
- "category": "AI",
+ "category": "Research",
"author": "Autonomous",
- "description": "Jev operates a web browser, live. On a made-up travel site it books flights in seconds: each step the page is written out as text and Jev, TypeSafe's System One model, picks one element from the list. Add pop-ups and layout shifts, or click the page yourself, and watch it cope.",
+ "description": "Point it at a web page and say what you want from each thing on it. A real Chrome opens, and Jev, TypeSafe's System One model, reads every page: one call judges every link and pulls every field. You get results.csv. Every value is text taken off the page, so nothing is invented.",
"engine": "claude",
"workspace": {
"template": "template",
- "marker": "site.json",
+ "marker": "browse.json",
"init": "toolchain/init-workspace.sh"
},
"agent": {
diff --git a/store/agents/jev-browser/skills/browser/SKILL.md b/store/agents/jev-browser/skills/browser/SKILL.md
index 8aea7ab8b..019aefc24 100644
--- a/store/agents/jev-browser/skills/browser/SKILL.md
+++ b/store/agents/jev-browser/skills/browser/SKILL.md
@@ -1,58 +1,87 @@
---
name: browser
-description: Write booking tasks and set the distraction dial for Jev Browser, then measure how many bookings come out exactly right and where layout shifts start to break it.
+description: Turn what a person wants off a website into a browse.json job, read what came back in results.csv, and sharpen the fields that came back thin.
---
-# Craft: Jev Browser runs
+# Craft: reading a website into a spreadsheet
-Jev Browser is a made-up travel site that Jev operates by itself, one picked element per step. You
-shape the run in `site.json`. The craft is a set of tasks that exercise every page, and a dial that
-shows the real limit: a click lands where the element WAS when the page was read.
+The person points at a page. You write the job. A real Chrome walks the site and Jev answers, per
+page, in one call. Every value in the spreadsheet is a piece of text that was really on the page,
+because Jev only ever picks from what the reader found. It cannot write a value, so it cannot
+invent one.
-## What Jev is asked, every step
+## The shape of a site
-One call, three questions, all answered from the same page text:
+Almost every site worth reading has two page kinds, and the job assumes them:
```
-action choice one option per element on the page right now (a new list every step)
-done noul "Is the task already complete?"
-obstacle noul "Is something covering the page that must be closed first?"
+a LIST page → links to each thing, and a link to the next list page
+a THING page → the details of one thing
```
-The page text is what a person could see: the task, the page name and URL, whether an overlay
-covers the page, the page's text, and every element that can be acted on with its label, its value
-and, for a result row's Select button, that row's details. The pane shows this text live.
+Point `start` at the list. If you point it at one thing's page, that one row is collected and the
+run ends. If the things have no page of their own (everything is on the list), say so to the person:
+this harness collects one row per page, so a list-only site gives one row for the whole list.
-## The funnel
+## Writing the fields
-`search` (From, To, Day, then Search flights) → `results` (pick the one right Select) →
-`details` (name, email, bags, no insurance, Continue) → `review` (no Flex upgrade, Confirm).
-With no distraction a booking takes about ten steps.
+A field is what to look for, in the words a person would use.
-The booking is right only if it is the exact flight the task describes (`cheapest`, `earliest` or
-`latest`, nonstop when asked), with the right name, email and bag count, and no extras.
+| What you want | Write |
+|---|---|
+| a value printed on the page | `{ "id": "rent", "name": "Rent", "ask": "the monthly rent" }` |
+| a judgement about the thing | `{ "ask": "Does it allow pets?", "type": "yesno" }` |
+| a place on your own scale | `{ "ask": "what condition it is in", "type": "score", "levels": ["needs work", "liveable", "newly done"] }` |
-## The dial
+- **Say it as it appears.** "the pay or salary range" beats "compensation" if the page says Salary.
+- **One value per field.** If the page prints "Acme · Leeds · £45,000" the reader offers that line
+ and each of its parts, so `company`, `place` and `salary` can each take their own part. A line
+ splits on ` · `, ` | `, a dash or a bullet, a label keeps its value (`UPC: a22124811bfa8350`), and
+ any number inside a line is offered alone.
+- **Ask for a number when you want to sort by one.** A page that says "In stock (19 available)"
+ offers both the sentence and `19`: ask for "how many copies are available" and you get the
+ number; ask for "the availability line" and you get the sentence. If the page never prints the
+ number on its own and Jev returns the sentence, that is right and honest. Derive the number with
+ a script afterwards and tell the person you did.
+- **A value that is not on the thing's page cannot be collected.** Rating stars drawn as pictures,
+ a price loaded after a click, a number inside an image: none of those are text. Check with the
+ reader (see AGENTS.md, "Look before you write the job") before promising a column.
+- **`yesno` is judgement, not text.** Use it for "is it remote?", not for "what is the salary?".
+ `check.mjs` warns when a `pick` field reads like a question.
-`distraction` adds, in order of pain: decoy buttons and links, a cookie wall on the first page,
-pop-ups that open between reading and clicking, and a sponsored row that pushes the results down so
-a click lands on the row above the one Jev chose. That last one makes a wrong booking that only a
-careful look at the next page can catch.
+## Reading what came back
-| feel | distraction |
-|---|---|
-| clean site, every booking right | 0 |
-| a normal annoying site (default) | 0.25 |
-| hostile | 0.6 |
-| everything moves | 1 |
+`results.csv`: one row per thing, every field with a confidence column. Then:
-## Verifying
+1. **Any column mostly empty?** The verdict marks it `thin`. Either the words are wrong, or the
+ value is not on those pages. Open one page with the reader and look.
+2. **Any column with low confidence?** Jev was choosing between two pieces of text. Read three of
+ those rows: usually two parts of one line, and a sharper ask fixes it.
+3. **Too few rows?** Look at the pane's link panel. If the real things scored under 0.5, `item` is
+ too vague: "a job posting" works, "an entry" does not. If it ran out, raise `maxItems`.
+4. **Rows that are not things?** A menu page scored over 0.5. Make `item` more specific, and say
+ what it is not: `"item": "a flat for rent, not a neighbourhood guide"`.
-```bash
-node "$JEV_DSH/toolchain/check.mjs"
-```
+Only then report. Count from the file with a script, never by eye.
+
+## What it costs and how long it takes
+
+One call per page. A list page with 120 links is about 122 questions in that one call; a thing's
+page is one question per field. Measured on 2026-09-20 with live Jev through OpenRouter: 24 things
+off a 36-item board took 33 s and $0.0023, and every one of the 144 cells matched the site's own
+data. The time is page loads, not Jev: expect about a second and a half a page.
+
+## The dial that is real
+
+Accuracy does not fall away as `maxItems` goes up; the tool is not fighting a budget, and the
+hundredth thing is read as carefully as the first. It is still a page load each, so ask for what the
+person needs. The honest limit is the **page**. A site that prints its values as text reads perfectly. A site that draws them, hides
+them behind a click, or loads them after a scroll gives blanks, and the blanks show up as a thin
+column rather than as a wrong answer. That is the tool telling the truth: it would rather leave a
+cell empty than write something the page never said.
+
+## Never
-Then watch the pane and read `.harness/verdict.json`: bookings, share exactly right, steps each,
-clicks lost to layout shifts, and the latest wrong bookings with the reason. Report the dial you
-chose and where the share that is right falls below 9 in 10. The pane's MOCK badge means the offline
-stand-in is driving, not the real model. Say so when you report.
+- Never go around a login wall, a block, a rate limit or a robots rule. Never ask for a password.
+- Never present the offline stand-in's rows as Jev's judgement.
+- Never retype a value from the page into the chat as if it were collected: run the job.
diff --git a/store/agents/jev-browser/store.json b/store/agents/jev-browser/store.json
index 30f751d29..748d7c1ad 100644
--- a/store/agents/jev-browser/store.json
+++ b/store/agents/jev-browser/store.json
@@ -2,18 +2,20 @@
"homepage": "https://typesafe.ai",
"upstream": "https://typesafe.ai",
"license": "MIT",
- "tagline": "Jev drives a web browser: a flight booked in seconds, one pick per step",
+ "tagline": "Point it at a web page, say what you want, get a spreadsheet",
"examples": [
{
- "prompt": "Write ten booking tasks for a family trip: different cities, days and bag counts, some nonstop only. Then raise distraction until fewer than 9 in 10 bookings come out exactly right, and tell me where that is.",
- "image": "https://raw.githubusercontent.com/autonomous-ai/openharness/main/store/showcase/jev-browser/booked-exactly-right.jpg",
- "caption": "A flight booked exactly as asked in 1.5 seconds and 10 steps, checked against the task"
+ "prompt": "Here is the search results page for two-bedroom flats in my area. Get me every listing with its address, monthly rent, how many bedrooms, and whether it says pets are allowed. Then tell me which five are the best value.",
+ "image": "https://raw.githubusercontent.com/autonomous-ai/openharness/main/store/showcase/jev-browser/reads-real-pages.jpg",
+ "caption": "One call judged all 16 links: the twelve real roles at 76-88%, the menu at 2%"
},
{
- "prompt": "Slow it down to one step a second with no distraction so I can read what Jev sees and what it picks at every step.",
- "image": "https://raw.githubusercontent.com/autonomous-ai/openharness/main/store/showcase/jev-browser/pick-the-flight.jpg",
- "caption": "Step mode: the page text Jev reads on the right, its chosen element ringed on the page"
+ "prompt": "Read every product in this category and put the name, price, stock count and product code in a spreadsheet. Stop after 40. Then tell me which ones are out of stock.",
+ "caption": "Every value is text taken off the page, with the confidence Jev had in it"
+ },
+ {
+ "prompt": "I have to check these 60 supplier pages every month for their lead times and minimum order. Write me a job I can run again, then run it and tell me what changed.",
+ "caption": "The job file is the recipe: run it next month and get next month's answer"
}
- ],
- "listed": false
+ ]
}
diff --git a/store/agents/jev-browser/template/browse.json b/store/agents/jev-browser/template/browse.json
new file mode 100644
index 000000000..620937074
--- /dev/null
+++ b/store/agents/jev-browser/template/browse.json
@@ -0,0 +1,17 @@
+{
+ "task": "Every open role on the job board, into a spreadsheet",
+ "start": "demo",
+ "item": "a job posting",
+ "fields": [
+ { "id": "title", "name": "Job title", "ask": "the job title" },
+ { "id": "company", "name": "Company", "ask": "the name of the company hiring" },
+ { "id": "salary", "name": "Salary", "ask": "the pay or salary range" },
+ { "id": "place", "name": "Location", "ask": "where the job is based" },
+ { "id": "posted", "name": "Posted", "ask": "the date it was posted" },
+ { "id": "remote", "name": "Remote?", "ask": "Can this job be done fully remotely?", "type": "yesno" }
+ ],
+ "maxItems": 12,
+ "maxPages": 4,
+ "sameSiteOnly": true,
+ "show": true
+}
diff --git a/store/agents/jev-browser/template/site.json b/store/agents/jev-browser/template/site.json
deleted file mode 100644
index 17c1c6d0f..000000000
--- a/store/agents/jev-browser/template/site.json
+++ /dev/null
@@ -1,17 +0,0 @@
-{
- "title": "Jev Browser",
- "description": "Jev books flights on a made-up travel site, one click at a time.",
- "site": "SkyHop",
- "stepMs": 170,
- "distraction": 0.25,
- "flights": 7,
- "maxSteps": 45,
- "seed": 11,
- "style": "Finish the task in as few actions as you can. Fill every field the task gives you, choose exactly the flight it asks for, refuse every extra, and close anything that gets in the way.",
- "tasks": [
- { "from": "SFO", "to": "JFK", "day": "Friday", "pick": "cheapest", "nonstop": true, "name": "Ada Park", "email": "[email protected]", "bags": 1 },
- { "from": "SEA", "to": "AUS", "day": "Monday", "pick": "earliest", "nonstop": false, "name": "Omar Reyes", "email": "[email protected]", "bags": 0 },
- { "from": "BOS", "to": "LAX", "day": "Sunday", "pick": "cheapest", "nonstop": false, "name": "Mei Tanaka", "email": "[email protected]", "bags": 2 },
- { "from": "DEN", "to": "MIA", "day": "Wednesday", "pick": "latest", "nonstop": true, "name": "Lena Okafor", "email": "[email protected]", "bags": 1 }
- ]
-}
diff --git a/store/agents/jev-browser/test/viewer.test.mjs b/store/agents/jev-browser/test/viewer.test.mjs
index 72349df58..effd7e9de 100644
--- a/store/agents/jev-browser/test/viewer.test.mjs
+++ b/store/agents/jev-browser/test/viewer.test.mjs
@@ -1,225 +1,271 @@
-// Integration tests for Jev Browser. They boot the real viewer on an ephemeral port against a temp
-// workspace and drive time with the `tick` control, so nothing here waits on a timer.
+// Jev Browser: the job file, the page reader, the guards, and a whole run against a real Chrome
+// over a real HTTP site served on the loopback. Chrome-dependent tests skip where there is no Chrome.
import { test } from 'node:test'
import assert from 'node:assert/strict'
-import { mkdtempSync, writeFileSync, readFileSync, existsSync } from 'node:fs'
+import { mkdtempSync, cpSync, readFileSync, existsSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
+import { connect } from 'node:net'
import { join, dirname } from 'node:path'
import { fileURLToPath } from 'node:url'
-import { execFileSync } from 'node:child_process'
-import net from 'node:net'
+import { normalizeJob, listQuestions, itemQuestions, rowFrom, pickLinks, LIMITS } from '../viewer/crawl.mjs'
+import { readPage, blockOptions, blockText, canonical, pageState } from '../viewer/page.mjs'
+import { openChrome, findChrome, checkUrl, Refused } from '../toolchain/chrome.mjs'
+import { startDemoSite } from '../viewer/demosite.mjs'
+import { startBrowserViewer } from '../viewer/viewer.mjs'
const HERE = dirname(fileURLToPath(import.meta.url))
-const ROOT = join(HERE, '..')
-const { startBrowserViewer, sanitize } = await import(join(ROOT, 'viewer/viewer.mjs'))
-const { DEFAULT, createWorld, observe, act, elements, bestFlight } = await import(join(ROOT, 'viewer/sim.mjs'))
-const { browserMock, readTask } = await import(join(ROOT, 'viewer/mock.mjs'))
-const TEMPLATE = JSON.parse(readFileSync(join(ROOT, 'template/site.json'), 'utf8'))
-
-async function boot(overrides = {}) {
- const ws = mkdtempSync(join(tmpdir(), 'jev-browser-test-'))
- writeFileSync(join(ws, 'site.json'), JSON.stringify({ ...TEMPLATE, ...overrides }))
- const viewer = await startBrowserViewer({ workspace: ws, port: 0 })
- const base = viewer.url
- const ctl = async (body) => { const r = await fetch(`${base}/control`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) }); return { status: r.status, json: await r.json() } }
- const state = async () => (await fetch(`${base}/state`)).json()
- await ctl({ cmd: 'pause' }); await ctl({ cmd: 'reset' })
- return { ws, viewer, base, ctl, state }
-}
-
-/** Run bookings straight on the sim with the stand-in: fast, no server. */
-function runBookings(cfg, n) {
- let ok = 0, steps = 0, mis = 0
- for (let i = 0; i < n; i++) {
- const w = createWorld(cfg, i, Math.floor(i / cfg.tasks.length))
- while (w.status === 'running') { const o = observe(w, cfg); act(w, cfg, browserMock(o.text, 'action', { options: Object.keys(o.options), descriptions: o.options }).choice) }
- if (w.result.ok) ok++; steps += w.steps; mis += w.misclicks
- }
- return { right: ok / n, steps: steps / n, misclicks: mis / n }
-}
+const TEMPLATE = join(HERE, '../template')
+const noChrome = findChrome() ? false : 'Google Chrome is not on this machine'
+process.env.JEV_BROWSER_HEADLESS = '1'
-test('Jev books a flight step by step, answering three questions per call', async () => {
- const v = await boot({ distraction: 0 })
- try {
- await v.ctl({ cmd: 'tick', n: 3 })
- let s = await v.state()
- assert.equal(s.steps, 3)
- assert.equal(s.page, 'search')
- assert.deepEqual(s.els.filter((e) => e.role === 'input').map((e) => e.value), ['SFO', 'JFK', 'Friday'])
- const sum = Object.values(s.last.probs).reduce((a, b) => a + b, 0)
- assert.ok(Math.abs(sum - 1) < 1e-6 && Math.max(...Object.values(s.last.probs)) < 0.999, 'a real distribution')
- await v.ctl({ cmd: 'tick', n: 7 })
- s = await v.state()
- assert.equal(s.status, 'done')
- assert.equal(s.result.ok, true, s.result.why)
- assert.equal(s.steps, 10)
- assert.equal(s.session.tasks, 1)
- const jev = await (await fetch(`${v.base}/jev`)).json()
- assert.equal(jev.calls, 10)
- assert.equal(jev.last.questions.length, 3)
- assert.ok(jev.costUsd > 0)
- } finally { await v.viewer.close() }
+/** fetch will not send a foreign Host header, so the loopback guard is tested over a raw socket. */
+const rawGet = (base, path, host) => new Promise((resolve, reject) => {
+ const socket = connect(Number(new URL(base).port), '127.0.0.1', () => socket.write(`GET ${path} HTTP/1.1\r\nHost: ${host}\r\nConnection: close\r\n\r\n`))
+ let data = ''
+ socket.on('data', (c) => { data += c })
+ socket.on('error', reject)
+ socket.on('end', () => resolve(Number(data.slice(9, 12))))
})
-test('it never ends: after a booking the next task starts by itself', async () => {
- const v = await boot({ distraction: 0, stepMs: 200 })
- try {
- await v.ctl({ cmd: 'tick', n: 10 })
- assert.equal((await v.state()).task.index, 0)
- await v.ctl({ cmd: 'tick', n: 14 }) // 2.6 s of rest at 200 ms a step, then the new task's first step
- const s = await v.state()
- assert.equal(s.task.index, 1)
- assert.match(s.task.goal, /SEA to AUS/)
- } finally { await v.viewer.close() }
-})
+const browser = async (allowedHosts = ['127.0.0.1', 'localhost']) =>
+ openChrome({ profileDir: mkdtempSync(join(tmpdir(), 'jev-browser-test-')), show: false, allowedHosts })
-test('site.json really takes effect, and pane overrides work', async () => {
- const tasks = [{ from: 'LIS', to: 'OSL', day: 'Tuesday', pick: 'latest', nonstop: false, name: 'Ines Costa', email: '[email protected]', bags: 3 }]
- const v = await boot({ title: 'Nordic Run', site: 'FjordAir', tasks, distraction: 0.6, stepMs: 400 })
- try {
- let s = await v.state()
- assert.equal(s.title, 'Nordic Run'); assert.equal(s.site, 'FjordAir')
- assert.equal(s.url, 'https://fjordair.example/')
- assert.equal(s.distraction, 0.6); assert.equal(s.stepMs, 400)
- assert.match(s.task.goal, /latest flight from LIS to OSL on Tuesday for Ines Costa \([email protected]\), 3 checked bags/)
- await v.ctl({ cmd: 'set', key: 'distraction', value: 0.1 })
- assert.equal((await v.state()).distraction, 0.1)
- await v.ctl({ cmd: 'set', key: 'nope', value: 5 })
- await v.ctl({ cmd: 'popup' })
- s = await v.state()
- assert.equal(s.overlay, 'popup')
- assert.match(s.stateText, /OVERLAY: a pop-up covers the page/)
- } finally { await v.viewer.close() }
+test('the job file: what it fills in, and what it says is wrong', () => {
+ const { job, errors } = normalizeJob(JSON.parse(readFileSync(join(TEMPLATE, 'browse.json'), 'utf8')))
+ assert.deepEqual(errors, [])
+ assert.equal(job.fields.length, 6)
+ assert.deepEqual(job.fields.map((f) => f.type), ['pick', 'pick', 'pick', 'pick', 'pick', 'yesno'])
+ assert.equal(job.sameSiteOnly, true)
+
+ const bare = normalizeJob({})
+ assert.match(bare.errors.join(' '), /no "start"/)
+ assert.match(bare.errors.join(' '), /no "fields"/)
+
+ const messy = normalizeJob({
+ start: 'https://shop.example.com/all', item: 'a product', alsoVisit: ['https://cdn.example.org/x'],
+ fields: ['the price', { ask: 'the name' }, { id: 'the price', ask: 'again' }, { ask: 'how new', type: 'score', levels: ['old'] }, { type: 'pick' }],
+ maxItems: 99999, maxPages: 0,
+ })
+ assert.deepEqual(messy.job.fields.map((f) => f.id), ['the_price', 'the_name'])
+ assert.match(messy.errors.join(' '), /both called "the_price"/)
+ assert.match(messy.errors.join(' '), /needs at least two levels/)
+ assert.match(messy.errors.join(' '), /a field needs "ask"/)
+ assert.equal(messy.job.maxItems, LIMITS.maxItems) // clamped, not obeyed
+ assert.equal(messy.job.maxPages, 1)
+ assert.deepEqual(messy.job.hosts, ['shop.example.com', 'cdn.example.org'])
})
-test('a person can click the page, and the click is a real step', async () => {
- const v = await boot({ distraction: 0 })
- try {
- await v.ctl({ cmd: 'click', id: 'to' })
- let s = await v.state()
- assert.equal(s.steps, 1)
- assert.equal(s.els.find((e) => e.id === 'to').value, 'JFK')
- assert.match(s.lastAction.note, /you clicked/)
- await v.ctl({ cmd: 'click', id: 'search-flights' }) // still disabled: a wasted step, the page does not move
- s = await v.state()
- assert.equal(s.page, 'search'); assert.equal(s.wasted, 1); assert.equal(s.shake, 'search-flights')
- await v.ctl({ cmd: 'tick', n: 12 })
- assert.equal((await v.state()).result.ok, true, 'Jev still finishes the booking')
- } finally { await v.viewer.close() }
+test('an address is checked before anything opens it', () => {
+ assert.equal(checkUrl('https://example.com/a?b=1', ['example.com']), 'https://example.com/a?b=1')
+ assert.equal(checkUrl('https://shop.example.com/a', ['example.com']), 'https://shop.example.com/a', 'a subdomain of a named site is the same site')
+ for (const bad of ['file:///etc/passwd', 'javascript:alert(1)', 'data:text/html,hi', 'chrome://settings', 'not a url']) {
+ assert.throws(() => checkUrl(bad, []), Refused, bad)
+ }
+ assert.throws(() => checkUrl('https://evil.example/x', ['example.com']), /not one of the sites this job named/)
+ assert.throws(() => checkUrl('https://notexample.com/x', ['example.com']), /not one of the sites/, 'a host that merely ends in the name is not the site')
})
-test('a progressive verdict is written', async () => {
- const v = await boot()
+test('the reader: links, values, and none of the page furniture', { skip: noChrome }, async () => {
+ const site = await startDemoSite()
+ const chrome = await browser()
try {
- await v.ctl({ cmd: 'tick', n: 40 })
- const verdict = JSON.parse(readFileSync(join(v.ws, '.harness/verdict.json'), 'utf8'))
- assert.equal(verdict.spec, 1); assert.equal(verdict.ready, true)
- assert.match(verdict.summary, /bookings · \d+% exactly right/)
- assert.ok(Array.isArray(verdict.findings) && Array.isArray(verdict.phases))
- assert.equal(verdict.artifact, 'site.json')
- } finally { await v.viewer.close() }
+ await chrome.go(site.url)
+ const list = await readPage(chrome)
+ assert.ok(list.links.length > 10)
+ assert.ok(list.links.some((l) => /\/job\/\d+/.test(l.path)), 'the job links are there')
+ assert.equal(new Set(list.links.map((l) => l.url)).size, list.links.length, 'one row per address')
+ assert.ok(!list.blocks.some((b) => /None of these roles exist/.test(b.text)), 'the footer is not offered as a value')
+ assert.ok(!list.blocks.some((b) => b.text === 'Fernhill Jobs'), 'the header is not offered as a value')
+
+ const job = list.links.find((l) => /\/job\/\d+/.test(l.path))
+ await chrome.go(job.url)
+ const page = await readPage(chrome)
+ const mashed = page.blocks.find((b) => b.parts.length)
+ assert.ok(mashed, 'a line that holds two values is also offered in parts')
+ assert.equal(mashed.parts.length, 2)
+ const options = blockOptions(page)
+ assert.equal(options[`b${mashed.n}p0`], mashed.parts[0])
+ assert.equal(blockText(page, `b${mashed.n}p1`), mashed.parts[1])
+ assert.equal(blockText(page, `b${mashed.n}`), mashed.text)
+ // Nothing can come back that was not on the page.
+ for (const made of ['none', 'b99999', 'b1p9', 'whatever', '', null]) assert.equal(blockText(page, made), '', String(made))
+ const state = pageState(page)
+ assert.ok(state.page_text.length > 40 && state.page_text.length <= 2500)
+ } finally { await chrome.close(); await site.close() }
})
-test('every control command answers 200, and junk is rejected', async () => {
- const v = await boot()
+test('the guards: it will not submit, will not press a danger word, will not type a secret', { skip: noChrome }, async () => {
+ const site = await startDemoSite()
+ const chrome = await browser()
try {
- for (const cmd of ['pause', 'tick', 'start', 'pause', 'popup', 'reset']) assert.equal((await v.ctl({ cmd })).status, 200, cmd)
- assert.equal((await v.ctl({ cmd: 'task', index: 2 })).status, 200)
- assert.match((await v.state()).task.goal, /BOS to LAX/)
- assert.equal((await v.ctl({ cmd: 'click', id: 'no-such-element' })).status, 200)
- assert.equal((await fetch(`${v.base}/control`, { method: 'POST', body: '{not json' })).status, 400)
- assert.equal((await fetch(`${v.base}/nope`)).status, 404)
- } finally { await v.viewer.close() }
+ await chrome.go(new URL('/login', site.url).toString())
+ const page = await readPage(chrome)
+ const submit = page.controls.find((c) => /sign in/i.test(c.label) && c.tag === 'button')
+ assert.ok(submit, 'the sign-in button is on the page')
+ await assert.rejects(() => chrome.click(submit.css, submit.label), /submits a form/)
+ // A danger word is refused on its own, before the page is even touched.
+ await chrome.evaluate(`(() => { const b = document.createElement('button'); b.textContent = 'Delete account'; b.setAttribute('data-jev-n', '900'); document.body.append(b) })()`)
+ await assert.rejects(() => chrome.click('[data-jev-n="900"]', 'Delete account'), /looks like it does something for real/)
+ await assert.rejects(() => chrome.click('[data-jev-n="900"]', 'Pay now'), Refused)
+ const password = page.controls.find((c) => c.type === 'password')
+ await assert.rejects(() => chrome.type(password.css, 'hunter2'), /never types those/)
+ const email = page.controls.find((c) => (c.label || '').toLowerCase().includes('email'))
+ assert.equal(await chrome.type(email.css, '[email protected]'), true, 'an ordinary field is fine')
+ // Off-site is refused even when asked directly.
+ await assert.rejects(() => chrome.go('https://example.com/'), /not one of the sites/)
+ await assert.rejects(() => chrome.open('file:///etc/passwd'), Refused)
+ } finally { await chrome.close(); await site.close() }
})
-test('a broken site.json keeps the demo alive and reports the problem', async () => {
- const v = await boot()
+test('the questions: one call holds every link and every field', { skip: noChrome }, async () => {
+ const site = await startDemoSite()
+ const chrome = await browser()
try {
- writeFileSync(join(v.ws, 'site.json'), '{ "title": "oops", ')
- await new Promise((r) => setTimeout(r, 250))
- await v.ctl({ cmd: 'tick', n: 5 })
- let s = await v.state()
- assert.match(String(s.cfgError), /site\.json/)
- assert.ok(s.steps >= 5)
- writeFileSync(join(v.ws, 'site.json'), JSON.stringify({ ...TEMPLATE, title: 'Back again', distraction: 99, tasks: 'nonsense' }))
- await new Promise((r) => setTimeout(r, 250))
- s = await v.state()
- assert.equal(s.cfgError, null); assert.equal(s.title, 'Back again')
- assert.equal(s.distraction, 1, 'out-of-range values are clamped')
- assert.equal(s.task.count, DEFAULT.tasks.length, 'a bad task list falls back to the built-in tasks')
- } finally { await v.viewer.close() }
+ const { job } = normalizeJob({ ...JSON.parse(readFileSync(join(TEMPLATE, 'browse.json'), 'utf8')), start: site.url })
+ await chrome.go(site.url)
+ const list = await readPage(chrome)
+ const { questions, links } = listQuestions(list, job)
+ assert.equal(Object.keys(questions).length, links.length + 2, 'one question per link, plus the kind and the next page')
+ assert.ok(links.length > 10)
+ assert.match(questions[`l${links[0].n}`].instructions, /is the text of a link on this page/)
+ // A made-up set of answers turns into the right links, in page order.
+ const answers = { nextpage: { choice: `l${links[links.length - 1].n}` } }
+ for (const [i, l] of links.entries()) answers[`l${l.n}`] = { noul: i % 2 ? 0.9 : 0.1 }
+ const got = pickLinks(links, answers)
+ assert.equal(got.items.length, Math.floor(links.length / 2))
+ assert.deepEqual(got.items.map((l) => l.n), [...got.items.map((l) => l.n)].sort((a, b) => a - b))
+ assert.equal(got.next.n, links[links.length - 1].n)
+ assert.equal(pickLinks(links, { nextpage: { choice: 'none' } }).next, null)
+
+ await chrome.go(links.find((l) => /\/job\//.test(l.path)).url)
+ const item = await readPage(chrome)
+ const iq = itemQuestions(item, job)
+ assert.equal(Object.keys(iq.questions).length, job.fields.length + 1)
+ const pick = item.blocks[0]
+ const made = { kind: { choice: 'item' }, f_title: { choice: `b${pick.n}`, confidence: 0.9 }, f_remote: { noul: 0.8 } }
+ const row = rowFrom(item, job, made)
+ assert.equal(row.fields.title, pick.text, 'the cell holds the exact text from the page')
+ assert.equal(row.fields.remote, 'yes')
+ assert.equal(row.confidence.remote, 0.8)
+ assert.equal(row.fields.salary, '', 'a field with no answer stays empty')
+ } finally { await chrome.close(); await site.close() }
})
-test('the viewer is loopback only', async () => {
- const v = await boot()
+test('a whole run: rows land in results.csv, the verdict says what happened', { skip: noChrome }, async () => {
+ process.env.JEV_OFFLINE = '1'
+ const site = await startDemoSite()
+ const ws = mkdtempSync(join(tmpdir(), 'jev-browser-run-'))
+ cpSync(TEMPLATE, ws, { recursive: true })
+ const cfg = JSON.parse(readFileSync(join(ws, 'browse.json'), 'utf8'))
+ writeFileSync(join(ws, 'browse.json'), JSON.stringify({ ...cfg, start: site.url, maxItems: 5, maxPages: 2 }))
+ const v = await startBrowserViewer({ workspace: ws, port: 0 })
+ const ctl = async (cmd, body = {}) => (await fetch(`${v.url}/control`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ cmd, ...body }) })).json()
try {
- const port = Number(new URL(v.base).port)
- const status = await new Promise((resolve) => {
- const sock = net.connect(port, '127.0.0.1', () => sock.write('GET /state HTTP/1.1\r\nHost: evil.example.com\r\nConnection: close\r\n\r\n'))
- let buf = ''
- sock.on('data', (d) => { buf += d })
- sock.on('end', () => resolve(Number((buf.match(/^HTTP\/1\.1 (\d+)/) ?? [])[1])))
- sock.on('error', () => resolve(0))
- })
- assert.equal(status, 403)
- } finally { await v.viewer.close() }
+ const before = await (await fetch(`${v.url}/state`)).json()
+ assert.equal(before.phase, 'idle')
+ assert.equal(before.client, 'mock')
+ assert.equal(before.chrome.found, true)
+ const run = await ctl('run')
+ assert.equal(run.phase, 'done')
+ assert.equal(run.rows, 5, 'it stops at maxItems')
+
+ const s = await (await fetch(`${v.url}/state`)).json()
+ assert.equal(s.rows.length, 5)
+ assert.ok(s.progress.links > 10, 'it judged every link on the list page')
+ assert.equal(s.progress.calls, 6, 'one call for the list page, one for each of the five items')
+ assert.ok(s.progress.questions > s.progress.calls * 3)
+ assert.equal(s.progress.errors, 0)
+ assert.ok(s.links.length > 10 && s.links.every((l) => l.p >= 0 && l.p <= 1))
+ for (const r of s.rows) assert.match(r.url, /\/job\/\d+$/, 'every row is one of the things, not a menu page')
+ assert.equal(new Set(s.rows.map((r) => r.url)).size, 5, 'no page is collected twice')
+
+ const csv = readFileSync(join(ws, 'results.csv'), 'utf8').trim().split('\n')
+ assert.equal(csv.length, 6)
+ assert.match(csv[0], /^item,page title,address,Job title,Job title confidence,/)
+ const verdict = JSON.parse(readFileSync(join(ws, '.harness/verdict.json'), 'utf8'))
+ assert.equal(verdict.ready, true)
+ assert.equal(verdict.run.rows, 5)
+ assert.equal(verdict.run.client, 'mock')
+ assert.equal(verdict.run.fields.length, 6)
+ assert.ok(verdict.findings.some((f) => /offline stand-in/.test(f.message)), 'it says the rows are not Jev\'s judgement')
+
+ // The download and the same-origin guard.
+ const dl = await fetch(`${v.url}/download/results.csv`)
+ assert.equal(dl.status, 200)
+ assert.equal((await dl.text()).split('\n').length, 7)
+ assert.equal((await fetch(`${v.url}/download/browse.json`)).status, 404)
+ assert.equal((await fetch(`${v.url}/control`, { method: 'POST', headers: { origin: 'https://evil.example', 'content-type': 'application/json' }, body: '{"cmd":"start"}' })).status, 403)
+
+ // Reset clears the rows and reads the job again.
+ assert.equal((await ctl('reset')).phase, 'idle')
+ assert.equal((await (await fetch(`${v.url}/state`)).json()).rows.length, 0)
+ assert.equal(readFileSync(join(ws, 'results.csv'), 'utf8').trim().split('\n').length, 1)
+
+ // The pane's address bar obeys the same rules as everything else.
+ const off = await ctl('openHere', { url: 'https://example.com/' })
+ assert.equal(off.ok, false)
+ assert.match(off.error, /not one of the sites/)
+ } finally { await v.close(); await site.close(); delete process.env.JEV_OFFLINE }
})
-test('the dial is honest: distraction costs steps and correct bookings', () => {
- const clean = runBookings(sanitize({ ...DEFAULT, distraction: 0 }), 200)
- const hostile = runBookings(sanitize({ ...DEFAULT, distraction: 0.9 }), 200)
- assert.equal(clean.right, 1); assert.equal(clean.steps, 10); assert.equal(clean.misclicks, 0)
- assert.ok(hostile.right < 0.92 && hostile.right > 0.5, `hostile right ${hostile.right}`)
- assert.ok(hostile.steps > clean.steps * 1.4, `hostile steps ${hostile.steps}`)
- assert.ok(hostile.misclicks > 1.5, `hostile misclicks ${hostile.misclicks}`)
+test('a broken job file keeps the pane alive and says what to fix', async () => {
+ const ws = mkdtempSync(join(tmpdir(), 'jev-browser-bad-'))
+ writeFileSync(join(ws, 'browse.json'), '{ "start": "https://example.com", "fields": [] }')
+ const v = await startBrowserViewer({ workspace: ws, port: 0 })
+ try {
+ const s = await (await fetch(`${v.url}/state`)).json()
+ assert.match(s.error, /no "fields"/)
+ const start = await (await fetch(`${v.url}/control`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{"cmd":"start"}' })).json()
+ assert.equal(start.ok, false, 'it will not run a job it cannot read')
+ const verdict = JSON.parse(readFileSync(join(ws, '.harness/verdict.json'), 'utf8'))
+ assert.equal(verdict.ready, false)
+ assert.ok(verdict.findings.some((f) => f.severity === 'error'))
+ } finally { await v.close() }
})
-test('a layout shift makes the click land on what is there now', () => {
- const cfg = sanitize({ ...DEFAULT, distraction: 1 })
- // Walk many bookings until a sponsored row shifts the results under a Select click.
- for (let i = 0; i < 80; i++) {
- const w = createWorld(cfg, i, 0)
- while (w.status === 'running') {
- const o = observe(w, cfg)
- const id = browserMock(o.text, 'action', { options: Object.keys(o.options), descriptions: o.options }).choice
- const rec = act(w, cfg, id)
- if (rec.shift === 'sponsored row' && id.startsWith('select-')) {
- assert.notEqual(rec.landed, id, 'the list moved down, so the click hit another element')
- assert.match(rec.note, /layout shifted/)
- assert.ok(elements(w, cfg).length > 0)
- return
- }
- }
- }
- assert.fail('no sponsored-row shift happened in 80 bookings at distraction 1')
+test('the pane files are served, and nothing else', async () => {
+ const ws = mkdtempSync(join(tmpdir(), 'jev-browser-files-'))
+ cpSync(TEMPLATE, ws, { recursive: true })
+ const v = await startBrowserViewer({ workspace: ws, port: 0 })
+ try {
+ for (const name of ['', 'studio.js', 'studio.css', 'base.css', 'jev-hud.js']) assert.equal((await fetch(`${v.url}/${name}`)).status, 200, name)
+ for (const name of ['viewer.mjs', 'crawl.mjs', 'page.mjs', '../toolchain/jev.mjs', 'demosite.mjs']) assert.equal((await fetch(`${v.url}/${name}`)).status, 404, name)
+ assert.equal(await rawGet(v.url, '/', 'example.com'), 403, 'a page on another name cannot reach this server')
+ assert.equal(await rawGet(v.url, '/', '127.0.0.1'), 200)
+ assert.equal((await (await fetch(`${v.url}/jev`)).json()).canConnect, true)
+ } finally { await v.close() }
})
-test('every task has exactly one right flight, and the stand-in reads only the text', () => {
- const cfg = sanitize(DEFAULT)
- for (let i = 0; i < 40; i++) {
- const w = createWorld(cfg, i, i)
- const best = bestFlight(w.task, w.flights)
- assert.ok(best, 'a right answer exists')
- if (w.task.nonstop) assert.equal(best.stops, 0)
- }
- const w = createWorld(sanitize({ ...DEFAULT, distraction: 0 }), 0, 0)
- const o = observe(w, sanitize({ ...DEFAULT, distraction: 0 }))
- assert.deepEqual(readTask(o.text), { pick: 'cheapest', nonstop: true, from: 'SFO', to: 'JFK', day: 'Friday', name: 'Ada Park', email: '[email protected]', bags: 1 })
- assert.equal(browserMock(o.text, 'action', { options: Object.keys(o.options), descriptions: o.options }).choice, 'from')
- assert.equal(browserMock('nothing to do with a browser', 'action', { options: ['a', 'b'] }), null)
+test('one address, one row: the same page under two names is not collected twice', () => {
+ assert.equal(canonical('https://a.com/x/?utm_source=z#frag'), 'https://a.com/x')
+ assert.equal(canonical('https://a.com/x'), canonical('https://a.com/x/'))
+ assert.equal(canonical('https://a.com/x?page=2'), 'https://a.com/x?page=2')
})
-test('check.mjs accepts the template and rejects bad values', () => {
- const run = (site) => {
- const ws = mkdtempSync(join(tmpdir(), 'jev-browser-check-'))
- writeFileSync(join(ws, 'site.json'), JSON.stringify(site))
- try { return { code: 0, out: execFileSync(process.execPath, [join(ROOT, 'toolchain/check.mjs')], { env: { ...process.env, HARNESS_WORKSPACE: ws }, encoding: 'utf8' }) } }
- catch (e) { return { code: e.status, out: String(e.stdout) } }
- }
- assert.equal(run(TEMPLATE).code, 0)
- const loud = run({ ...TEMPLATE, distraction: 3 })
- assert.equal(loud.code, 1); assert.match(loud.out, /distraction/)
- const badTask = run({ ...TEMPLATE, tasks: [{ ...TEMPLATE.tasks[0], pick: 'fanciest', bags: 9 }] })
- assert.equal(badTask.code, 1); assert.match(badTask.out, /pick must be/); assert.match(badTask.out, /bags must be/)
- assert.ok(existsSync(join(ROOT, 'LICENSE')))
+test('a label keeps its value, and a number in a sentence is offered on its own', { skip: noChrome }, async () => {
+ // The demo site prints its details as <dt>Salary</dt><dd>£35,000 - £55,000</dd>. Each distinct
+ // string is offered once, so without pairing the value is dropped as a repeat of the one on the
+ // list page and the label is left alone, looking like the page never said it.
+ const site = await startDemoSite()
+ const chrome = await browser()
+ try {
+ await chrome.go(new URL('/job/1000', site.url).toString())
+ const page = await readPage(chrome)
+ const texts = page.blocks.map((b) => b.text)
+ assert.ok(!texts.includes('Salary'), 'a label is never offered on its own')
+ const salary = page.blocks.find((b) => /^Salary: /.test(b.text))
+ assert.ok(salary, `the label and its value are one block: ${JSON.stringify(texts)}`)
+ assert.match(salary.parts[0], /^£[\d,]+ - £[\d,]+$/, 'the value alone is the first part')
+ const posted = page.blocks.find((b) => /^Posted: /.test(b.text))
+ assert.ok(posted && posted.parts.some((p) => /^\d{1,2} \w+ \d{4}$/.test(p)), 'the date alone is offered')
+ // A number inside a sentence is offered on its own, so a column can be sorted.
+ await chrome.evaluate(`(() => { const p = document.createElement('p'); p.textContent = 'In stock (19 available)'; document.querySelector('main').append(p) })()`)
+ const again = await readPage(chrome)
+ const stock = again.blocks.find((b) => b.text === 'In stock (19 available)')
+ assert.ok(stock, 'the sentence is offered')
+ assert.ok(stock.parts.includes('19'), `and the number on its own: ${JSON.stringify(stock.parts)}`)
+ assert.equal(blockText(again, `b${stock.n}p${stock.parts.indexOf('19')}`), '19')
+ // A choice takes at most 255 options, and parts push the count up.
+ assert.ok(Object.keys(blockOptions(again)).length <= 250)
+ } finally { await chrome.close(); await site.close() }
})
diff --git a/store/agents/jev-browser/toolchain/check.mjs b/store/agents/jev-browser/toolchain/check.mjs
index 8c54e78de..853db904d 100644
--- a/store/agents/jev-browser/toolchain/check.mjs
+++ b/store/agents/jev-browser/toolchain/check.mjs
@@ -1,49 +1,39 @@
#!/usr/bin/env node
-// check.mjs — validate a site.json for the Jev Browser harness: every number's range and every
-// task's shape. Prints one line per problem and exits 1 on any error.
+// check.mjs — validate a browse.json for the Jev Browser harness. The viewer and this script read
+// the job through the same code, so what passes here is what the pane will run.
+// node toolchain/check.mjs checks $HARNESS_WORKSPACE/browse.json (or ./browse.json)
+// node toolchain/check.mjs path.json checks that file
import { readFileSync } from 'node:fs'
import { join } from 'node:path'
+import { normalizeJob, LIMITS } from '../viewer/crawl.mjs'
+import { checkUrl, findChrome } from './chrome.mjs'
-const ws = process.env.HARNESS_WORKSPACE || '.'
-let errors = 0
-const error = (msg) => { errors++; console.log(`error ${msg}`) }
-const warn = (msg) => console.log(`warn ${msg}`)
+const file = process.argv[2] || join(process.env.HARNESS_WORKSPACE || '.', 'browse.json')
+let errors = 0, warnings = 0
+const error = (m) => { errors++; console.log(`error ${m}`) }
+const warn = (m) => { warnings++; console.log(`warn ${m}`) }
-let p
-try {
- p = JSON.parse(readFileSync(join(ws, 'site.json'), 'utf8'))
-} catch (e) {
- console.log(`error cannot read site.json: ${e.message}`)
- process.exit(1)
-}
+let raw
+try { raw = JSON.parse(readFileSync(file, 'utf8')) } catch (e) { console.log(`error cannot read browse.json: ${e.message}`); process.exit(1) }
+if (!raw || typeof raw !== 'object' || Array.isArray(raw)) { console.log('error browse.json must be a JSON object'); process.exit(1) }
-const range = (key, lo, hi, what) => {
- if (p[key] === undefined) return
- if (typeof p[key] !== 'number' || !Number.isFinite(p[key]) || p[key] < lo || p[key] > hi) error(`${key} (${what}) must be a number from ${lo} to ${hi}`)
-}
-range('stepMs', 40, 2000, 'milliseconds per Jev decision')
-range('distraction', 0, 1, 'the dial')
-range('flights', 4, 9, 'result rows per search')
-range('maxSteps', 12, 200, 'steps before a task fails')
-range('seed', 0, 1e9, 'random seed')
+const { job, errors: problems } = normalizeJob(raw)
+for (const p of problems) error(p)
-if (!Array.isArray(p.tasks) || p.tasks.length < 1 || p.tasks.length > 40) error('tasks must be a list of 1 to 40 bookings')
-else p.tasks.forEach((t, i) => {
- const at = `tasks[${i}]`
- for (const k of ['from', 'to', 'day', 'name', 'email']) if (typeof t?.[k] !== 'string' || !t[k].trim()) error(`${at}.${k} must be a non-empty string`)
- if (typeof t?.from === 'string' && !/^[A-Za-z]{2,4}$/.test(t.from.trim())) error(`${at}.from must be an airport code of 2 to 4 letters`)
- if (typeof t?.to === 'string' && !/^[A-Za-z]{2,4}$/.test(t.to.trim())) error(`${at}.to must be an airport code of 2 to 4 letters`)
- if (typeof t?.from === 'string' && t.from === t?.to) error(`${at}: from and to are the same airport`)
- if (!['cheapest', 'earliest', 'latest'].includes(t?.pick)) error(`${at}.pick must be "cheapest", "earliest" or "latest"`)
- if (typeof t?.nonstop !== 'boolean') error(`${at}.nonstop must be true or false`)
- if (!Number.isInteger(t?.bags) || t.bags < 0 || t.bags > 3) error(`${at}.bags must be a whole number from 0 to 3`)
- if (typeof t?.email === 'string' && !/@example\.(com|org|net)$/.test(t.email.trim())) warn(`${at}.email: use a made-up example.com address`)
- if (typeof t?.name === 'string' && /[()]/.test(t.name)) error(`${at}.name must not contain brackets`)
-})
-if (Array.isArray(p.tasks) && new Set(p.tasks.map((t) => t?.pick)).size === 1 && p.tasks.length > 2) warn('every task uses the same pick rule: mix cheapest, earliest and latest')
-if (!p.title || typeof p.title !== 'string') warn('no title')
-if (!p.style || typeof p.style !== 'string') warn('no style line: tell Jev how to work')
-else if (p.style.length > 600) error('style must be 600 characters or fewer')
+// "demo" is the little made-up job board the viewer serves itself.
+if (job.start && !/^demo$/i.test(job.start)) {
+ try { checkUrl(job.start, []) } catch (e) { error(`"start": ${e.message}`) }
+ if (/^http:\/\//i.test(job.start) && !/^http:\/\/(127\.0\.0\.1|localhost)/i.test(job.start)) warn('"start" is plain http, not https')
+}
+for (const f of job.fields) {
+ if (f.type === 'pick' && f.ask.length < 4) warn(`field "${f.id}": say what to look for in a few more words`)
+ if (f.type === 'pick' && /^(is|are|does|do|can|has|have|will|should)\b/i.test(f.ask)) warn(`field "${f.id}" reads as a yes or no question. Add "type": "yesno", or it will pick a piece of text instead`)
+}
+if (job.fields.length && !job.fields.some((f) => f.type === 'pick')) warn('no field takes its value off the page, so the spreadsheet will hold only judgements')
+if (job.maxItems > 200) warn(`maxItems is ${job.maxItems}: that is ${job.maxItems} page loads, so it will take a while`)
+if (!job.sameSiteOnly) warn('"sameSiteOnly": false lets it follow links off the site it started on')
+if (!findChrome()) warn('Google Chrome was not found on this machine. Install it, or set CHROME_PATH')
-console.log(errors ? `fail site.json has ${errors} error${errors > 1 ? 's' : ''}` : 'ok site.json is valid')
-process.exit(errors ? 1 : 0)
+if (errors) { console.log(`fail invalid browse.json (${errors} error${errors > 1 ? 's' : ''}, ${warnings} warning${warnings === 1 ? '' : 's'})`); process.exit(1) }
+console.log(`ok browse.json is valid: ${job.fields.length} field${job.fields.length === 1 ? '' : 's'}, up to ${job.maxItems} things from ${job.maxPages} pages${warnings ? `, ${warnings} warning${warnings > 1 ? 's' : ''}` : ''}`)
+console.log(`info limits: ${LIMITS.fields} fields, ${LIMITS.linksPerCall} links judged per call`)
diff --git a/store/agents/jev-browser/toolchain/chrome.mjs b/store/agents/jev-browser/toolchain/chrome.mjs
new file mode 100644
index 000000000..72251810f
--- /dev/null
+++ b/store/agents/jev-browser/toolchain/chrome.mjs
@@ -0,0 +1,206 @@
+// chrome.mjs — drive a real Chrome, with no dependencies (Node 22+ has WebSocket built in).
+//
+// The browser is the person's own: a visible window with a profile kept in the workspace, so a site
+// they log into once stays logged in for the next run. The harness never types a password.
+//
+// What this file will NOT do, whatever it is asked:
+// * leave http/https, or reach a host the job did not name
+// * submit a form, or click something that reads like pay, buy, delete or send
+// * type into a password, card or one-time-code field
+// Those refusals live here, next to the only code that can touch the page, so nothing above can
+// route around them.
+import { spawn } from 'node:child_process'
+import { createServer } from 'node:net'
+import { existsSync, mkdirSync, rmSync } from 'node:fs'
+import { platform } from 'node:os'
+
+export const CHROME_PATHS = {
+ darwin: ['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', '/Applications/Chromium.app/Contents/MacOS/Chromium', '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary'],
+ linux: ['/usr/bin/google-chrome', '/usr/bin/google-chrome-stable', '/usr/bin/chromium', '/usr/bin/chromium-browser', '/snap/bin/chromium'],
+ win32: ['C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe', 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe'],
+}
+
+/** Where Chrome is on this machine, or null. `CHROME_PATH` wins. */
+export function findChrome() {
+ const envPath = process.env.CHROME_PATH
+ if (envPath) return existsSync(envPath) ? envPath : null
+ for (const p of CHROME_PATHS[platform()] ?? []) if (existsSync(p)) return p
+ return null
+}
+
+const freePort = () => new Promise((resolve, reject) => {
+ const s = createServer()
+ s.once('error', reject)
+ s.listen(0, '127.0.0.1', () => { const { port } = s.address(); s.close(() => resolve(port)) })
+})
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms))
+
+/** A click on one of these is a door that does not open twice. Jev is never given the chance. */
+const DANGER = /\b(pay|buy|purchase|checkout|order now|place order|subscribe|donate|confirm and pay|delete|remove|deactivate|close account|unsubscribe|send|post|publish|submit|apply now|book now|reserve|sign out|log out)\b/i
+const SECRET_FIELD = /pass|pwd|secret|otp|2fa|one.?time|cvv|cvc|card|iban|account.?number|ssn|social.?security|pin\b/i
+
+export class Refused extends Error {}
+
+/** Only http(s), and only a host the job named. */
+export function checkUrl(raw, allowedHosts) {
+ let u
+ try { u = new URL(String(raw)) } catch { throw new Refused(`"${String(raw).slice(0, 80)}" is not a web address`) }
+ if (u.protocol !== 'http:' && u.protocol !== 'https:') throw new Refused(`${u.protocol} is not a web address this harness opens`)
+ if (allowedHosts?.length) {
+ const host = u.hostname.toLowerCase()
+ const ok = allowedHosts.some((h) => host === h || host.endsWith('.' + h))
+ if (!ok) throw new Refused(`${u.hostname} is not one of the sites this job named (${allowedHosts.join(', ')})`)
+ }
+ return u.toString()
+}
+
+/**
+ * Open a Chrome and talk to it.
+ * @param {object} o
+ * @param {string} o.profileDir kept between runs, so a login lasts
+ * @param {boolean} [o.show] a window the person can watch and take over (false for tests)
+ * @param {string[]} [o.allowedHosts]
+ * @param {number} [o.width] [o.height]
+ */
+export async function openChrome({ profileDir, show = true, allowedHosts = [], width = 1280, height = 900, chromePath = findChrome(), fresh = false } = {}) {
+ if (!chromePath) throw new Error('Google Chrome was not found. Install it, or set CHROME_PATH to where it is.')
+ if (fresh) rmSync(profileDir, { recursive: true, force: true })
+ mkdirSync(profileDir, { recursive: true })
+ const port = await freePort()
+ const args = [
+ `--remote-debugging-port=${port}`, `--user-data-dir=${profileDir}`,
+ '--no-first-run', '--no-default-browser-check', '--disable-background-networking',
+ '--disable-features=Translate,MediaRouter,OptimizationHints', '--disable-popup-blocking',
+ `--window-size=${width},${height}`, 'about:blank',
+ ]
+ if (!show) args.unshift('--headless=new')
+ const proc = spawn(chromePath, args, { stdio: 'ignore', detached: false })
+ const dead = new Promise((_, reject) => proc.once('exit', (code) => reject(new Error(`Chrome closed (code ${code})`))))
+
+ // Wait for the debugging port. Chrome takes a moment on a cold profile.
+ let version = null
+ for (let i = 0; i < 100 && !version; i++) {
+ try { version = await (await fetch(`http://127.0.0.1:${port}/json/version`)).json() } catch { await sleep(100) }
+ }
+ if (!version) { proc.kill(); throw new Error('Chrome started but never opened its debugging port') }
+
+ const target = await (await fetch(`http://127.0.0.1:${port}/json/new?about:blank`, { method: 'PUT' })).json()
+ const ws = new WebSocket(target.webSocketDebuggerUrl)
+ await Promise.race([new Promise((r, j) => { ws.onopen = r; ws.onerror = () => j(new Error('Chrome refused the debugging connection')) }), dead])
+
+ let nextId = 1
+ const pending = new Map()
+ const events = new Map() // method -> Set(handler)
+ ws.onmessage = (ev) => {
+ const m = JSON.parse(ev.data)
+ if (m.id && pending.has(m.id)) { const { r, j } = pending.get(m.id); pending.delete(m.id); m.error ? j(new Error(m.error.message)) : r(m.result) }
+ else if (m.method) for (const h of events.get(m.method) ?? []) h(m.params)
+ }
+ ws.onclose = () => { for (const { j } of pending.values()) j(new Error('Chrome went away')); pending.clear() }
+ const on = (method, handler) => { if (!events.has(method)) events.set(method, new Set()); events.get(method).add(handler); return () => events.get(method).delete(handler) }
+ const send = (method, params = {}, timeoutMs = 30000) => new Promise((resolve, reject) => {
+ if (ws.readyState !== 1) return reject(new Error('Chrome is not connected'))
+ const id = nextId++
+ const timer = setTimeout(() => { pending.delete(id); reject(new Error(`Chrome did not answer ${method} in time`)) }, timeoutMs)
+ pending.set(id, { r: (v) => { clearTimeout(timer); resolve(v) }, j: (e) => { clearTimeout(timer); reject(e) } })
+ ws.send(JSON.stringify({ id, method, params }))
+ })
+
+ await send('Page.enable')
+ await send('Runtime.enable')
+ await send('Network.enable')
+ await send('Page.setLifecycleEventsEnabled', { enabled: true })
+ // A download would put a file on the person's disk without them asking. Refuse them all.
+ await send('Browser.setDownloadBehavior', { behavior: 'deny' }).catch(() => {})
+
+ let lastStatus = null
+ on('Network.responseReceived', (p) => { if (p.type === 'Document') lastStatus = p.response.status })
+
+ async function evaluate(expression, { awaitPromise = true, timeoutMs = 20000 } = {}) {
+ const res = await send('Runtime.evaluate', { expression, awaitPromise, returnByValue: true }, timeoutMs)
+ if (res.exceptionDetails) throw new Error(res.exceptionDetails.exception?.description ?? res.exceptionDetails.text)
+ return res.result.value
+ }
+
+ /** Wait until the page has stopped loading, or the time runs out. Never throws. */
+ async function settle(ms = 12000) {
+ const end = Date.now() + ms
+ let lastHtmlSize = -1, still = 0
+ while (Date.now() < end) {
+ await sleep(250)
+ let state
+ try { state = await evaluate('[document.readyState, document.documentElement.innerHTML.length]', { timeoutMs: 5000 }) } catch { continue }
+ if (state?.[0] !== 'complete') { still = 0; continue }
+ if (state[1] === lastHtmlSize) { if (++still >= 2) return true } else still = 0
+ lastHtmlSize = state[1]
+ }
+ return false
+ }
+
+ const api = {
+ port,
+ /** Which hosts this session may reach. The job sets it; nothing else changes it. */
+ allowedHosts,
+ async go(url) {
+ const safe = checkUrl(url, api.allowedHosts)
+ lastStatus = null
+ await send('Page.navigate', { url: safe })
+ await settle()
+ return { url: await evaluate('location.href'), status: lastStatus }
+ },
+ evaluate,
+ settle,
+ async url() { return evaluate('location.href') },
+ async title() { return evaluate('document.title') },
+ /**
+ * Open one link. A link is followed by going to its address, not by clicking: a click can fire a
+ * script that buys something. The address is checked like any other.
+ */
+ async open(href) { return api.go(href) },
+ /** Click an element the page reader found, by the css path it gave. Refused on a danger word. */
+ async click(cssPath, label = '') {
+ if (DANGER.test(label)) throw new Refused(`"${label.trim().slice(0, 60)}" looks like it does something for real. This harness only reads pages.`)
+ const box = await evaluate(`(() => { const e = document.querySelector(${JSON.stringify(cssPath)}); if (!e) return null
+ const t = (e.tagName || '').toLowerCase(), ty = (e.getAttribute('type') || '').toLowerCase()
+ if (t === 'form' || ty === 'submit' || ty === 'image' || (t === 'button' && (e.type === 'submit' || e.form))) return 'submit'
+ if (e.closest('form') && (t === 'button' || t === 'input')) return 'submit'
+ e.scrollIntoView({ block: 'center' }); const r = e.getBoundingClientRect()
+ return r.width < 1 || r.height < 1 ? null : [r.x + r.width / 2, r.y + r.height / 2] })()`)
+ if (box === 'submit') throw new Refused('that control submits a form, and this harness only reads pages')
+ if (!box) throw new Refused('that element is not on the page any more')
+ for (const type of ['mouseMoved', 'mousePressed', 'mouseReleased']) {
+ await send('Input.dispatchMouseEvent', { type, x: box[0], y: box[1], button: 'left', clickCount: type === 'mouseMoved' ? 0 : 1 })
+ }
+ await settle(8000)
+ return { url: await evaluate('location.href') }
+ },
+ /** Type into a field. A field that could hold a secret is refused, whoever asks. */
+ async type(cssPath, text) {
+ const kind = await evaluate(`(() => { const e = document.querySelector(${JSON.stringify(cssPath)}); if (!e) return null
+ return [(e.getAttribute('type') || '').toLowerCase(), (e.getAttribute('name') || '') + ' ' + (e.getAttribute('id') || '') + ' ' + (e.getAttribute('autocomplete') || '') + ' ' + (e.getAttribute('aria-label') || '') + ' ' + (e.getAttribute('placeholder') || '')] })()`)
+ if (!kind) throw new Refused('that field is not on the page any more')
+ if (kind[0] === 'password' || SECRET_FIELD.test(kind[1]) || SECRET_FIELD.test(kind[0])) {
+ throw new Refused('that field wants a password, a card or a code. This harness never types those. Type it yourself in the browser window.')
+ }
+ await evaluate(`(() => { const e = document.querySelector(${JSON.stringify(cssPath)}); e.focus(); e.value = ''; })()`)
+ await send('Input.insertText', { text: String(text).slice(0, 400) })
+ await evaluate(`(() => { const e = document.querySelector(${JSON.stringify(cssPath)}); e.dispatchEvent(new Event('input', { bubbles: true })); e.dispatchEvent(new Event('change', { bubbles: true })) })()`)
+ return true
+ },
+ async back() { await send('Page.navigate', { url: 'javascript:history.back()' }).catch(() => {}); await evaluate('history.back()').catch(() => {}); await settle(8000); return evaluate('location.href') },
+ async shot({ quality = 55, maxWidth = 900 } = {}) {
+ const res = await send('Page.captureScreenshot', { format: 'jpeg', quality, captureBeyondViewport: false }, 15000)
+ return { jpegBase64: res.data, maxWidth }
+ },
+ on,
+ async close() {
+ try { ws.close() } catch { /* already gone */ }
+ try { await fetch(`http://127.0.0.1:${port}/json/close/${target.id}`).catch(() => {}) } catch { /* fine */ }
+ try { proc.kill() } catch { /* already gone */ }
+ await sleep(80)
+ try { proc.kill('SIGKILL') } catch { /* gone */ }
+ },
+ get alive() { return ws.readyState === 1 && proc.exitCode === null },
+ }
+ return api
+}
diff --git a/store/agents/jev-browser/toolchain/doctor.sh b/store/agents/jev-browser/toolchain/doctor.sh
index c74f27245..a88fc0c65 100755
--- a/store/agents/jev-browser/toolchain/doctor.sh
+++ b/store/agents/jev-browser/toolchain/doctor.sh
@@ -1,11 +1,12 @@
#!/usr/bin/env bash
-# Doctor: verify the runtime prerequisites for the Jev Browser viewer.
+# Doctor: what this harness needs to drive a real browser.
set -euo pipefail
-if ! command -v node >/dev/null 2>&1; then echo "fail node is required (Node 18+)"; exit 1; fi
-if ! node -e "process.exit(Number(process.versions.node.split('.')[0]) >= 18 ? 0 : 1)" 2>/dev/null; then
- echo "fail node >= 18 required (found $(node -v))"; exit 1
+if ! command -v node >/dev/null 2>&1; then echo "fail node is required (Node 22+)"; exit 1; fi
+if ! node -e "process.exit(Number(process.versions.node.split('.')[0]) >= 22 ? 0 : 1)" 2>/dev/null; then
+ echo "fail node >= 22 required for the built-in WebSocket (found $(node -v))"; exit 1
fi
echo "ok node $(node -v)"
cd "$(dirname "$0")/.."
+node -e "import('./toolchain/chrome.mjs').then((m) => { const p = m.findChrome(); console.log(p ? 'ok Google Chrome: ' + p : 'fail Google Chrome was not found. Install it, or set CHROME_PATH to where it is.'); process.exit(p ? 0 : 1) })"
node -e "import('./toolchain/jev.mjs').then((m) => console.log('ok Jev: ' + m.describeCredentials())).catch((e) => console.log('warn Jev client did not load: ' + e.message))"
echo "done"
diff --git a/store/agents/jev-browser/toolchain/viewer.sh b/store/agents/jev-browser/toolchain/viewer.sh
index 269edcb69..898d9745a 100755
--- a/store/agents/jev-browser/toolchain/viewer.sh
+++ b/store/agents/jev-browser/toolchain/viewer.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
-# Launch the Jev Browser viewer (Jev operates a made-up travel site, one picked element per step).
+# Launch the Jev Browser viewer (a real Chrome, driven by Jev, reading pages into results.csv).
set -euo pipefail
cd "$(dirname "$0")/.."
HARNESS_DSH_DIR="${HARNESS_DSH_DIR:-$PWD}"
diff --git a/store/agents/jev-browser/viewer/crawl.mjs b/store/agents/jev-browser/viewer/crawl.mjs
new file mode 100644
index 000000000..9c12cc3f3
--- /dev/null
+++ b/store/agents/jev-browser/viewer/crawl.mjs
@@ -0,0 +1,199 @@
+// crawl.mjs — the loop. One Jev call per page does all of the thinking about that page.
+//
+// On a list page that one call is: "is this page a list or one item?", plus a separate yes/no for
+// EVERY link on the page ("is this a link to a job posting?"), plus "which link is the next page?".
+// A hundred and twenty typed questions come back together, each with its own probability, because
+// asking Jev many questions about one state costs about the same as asking one.
+//
+// On an item page the same single call pulls every field: each field is a choice over the numbered
+// text blocks the reader found, so the answer IS a piece of text from the page.
+import { jev } from '../toolchain/jev.mjs'
+import { readPage, pageState, blockOptions, blockText, canonical } from './page.mjs'
+
+export const LIMITS = { fields: 12, maxPages: 500, maxItems: 2000, linksPerCall: 120 }
+
+const clamp = (v, lo, hi, d) => (Number.isFinite(Number(v)) ? Math.min(hi, Math.max(lo, Math.floor(Number(v)))) : d)
+const short = (s, n) => String(s ?? '').replace(/\s+/g, ' ').trim().slice(0, n)
+
+/** Read the job, fill in what it does not say, and list what is wrong with it. */
+export function normalizeJob(raw) {
+ const errors = []
+ const j = raw && typeof raw === 'object' && !Array.isArray(raw) ? raw : {}
+ const start = short(j.start, 500)
+ if (!start) errors.push('no "start": give the web address of the page to begin on')
+ const item = short(j.item, 160) || 'one of the things to collect'
+ const fields = []
+ const seen = new Set()
+ for (const f of Array.isArray(j.fields) ? j.fields : []) {
+ if (fields.length >= LIMITS.fields) { errors.push(`only the first ${LIMITS.fields} fields are used`); break }
+ const ask = short(typeof f === 'string' ? f : f?.ask, 200)
+ if (!ask) { errors.push('a field needs "ask": what to look for, in plain words'); continue }
+ const id = (short(typeof f === 'object' ? f?.id : '', 40) || ask).toLowerCase().replace(/[^a-z0-9]+/g, '_').replace(/^_+|_+$/g, '').slice(0, 32) || `f${fields.length + 1}`
+ if (seen.has(id)) { errors.push(`two fields are both called "${id}"`); continue }
+ seen.add(id)
+ const type = ['pick', 'yesno', 'score'].includes(f?.type) ? f.type : 'pick'
+ const levels = Array.isArray(f?.levels) ? f.levels.map((l) => short(l, 40)).filter(Boolean).slice(0, 10) : []
+ if (type === 'score' && levels.length < 2) { errors.push(`field "${id}" is a score, so it needs at least two levels`); continue }
+ fields.push({ id, ask, type, levels, name: short(typeof f === 'object' ? f?.name : '', 40) || id })
+ }
+ if (!fields.length) errors.push('no "fields": say what to collect for each item, in plain words')
+ let hosts = []
+ try { hosts = [new URL(start).hostname.toLowerCase()] } catch { /* the start address is already reported */ }
+ for (const h of Array.isArray(j.alsoVisit) ? j.alsoVisit : []) { const s = short(h, 200).toLowerCase().replace(/^https?:\/\//, '').split('/')[0]; if (s) hosts.push(s) }
+ return {
+ job: {
+ task: short(j.task, 200) || `Collect ${item}`,
+ start, item, fields,
+ keep: short(j.keep, 300),
+ maxPages: clamp(j.maxPages, 1, LIMITS.maxPages, 25),
+ maxItems: clamp(j.maxItems, 1, LIMITS.maxItems, 60),
+ sameSiteOnly: j.sameSiteOnly !== false,
+ show: j.show !== false,
+ hosts: [...new Set(hosts)],
+ },
+ errors,
+ }
+}
+
+// ---- the questions ------------------------------------------------------------------------------
+const KIND = { list: 'a page that lists many of them, with a link to each', item: 'the page of ONE of them, with its details', other: 'neither: a home page, a login wall, a search box, an error, or something else' }
+
+/** Everything worth asking about a list page, in one call. */
+export function listQuestions(page, job, { wantLinks = true, wantNext = true } = {}) {
+ const q = { kind: jev.choice(KIND, `The task is: ${job.task}. Each thing being collected is ${job.item}. What kind of page is this?`) }
+ const links = page.links.slice(0, LIMITS.linksPerCall)
+ if (wantLinks) for (const l of links) {
+ q[`l${l.n}`] = jev.noul(`"${l.label}" is the text of a link on this page, pointing at ${l.path}. Is "${l.label}" the title of ${job.item}, rather than part of the site's own menu?`, { true: `"${l.label}" names ${job.item}`, false: "it is a menu item, a filter, a category, a page number, or one of the site's own pages" })
+ }
+ if (wantNext) {
+ const options = { none: 'there is no next page link' }
+ for (const l of links) options[`l${l.n}`] = `"${l.label}" → ${l.path}`
+ q.nextpage = jev.choice(options, 'Which link goes to the NEXT PAGE of this same list (page 2, "next", "more results")? Not a link to one of the things themselves.')
+ }
+ return { questions: q, links }
+}
+
+/** Everything worth asking about one item's page, in one call. */
+export function itemQuestions(page, job) {
+ const options = blockOptions(page, { none: `this page does not say it` })
+ const q = { kind: jev.choice(KIND, `The task is: ${job.task}. Each thing being collected is ${job.item}. What kind of page is this?`) }
+ for (const f of job.fields) {
+ if (f.type === 'yesno') q[`f_${f.id}`] = jev.noul(`About ${job.item} on this page: ${f.ask}`)
+ else if (f.type === 'score') q[`f_${f.id}`] = jev.score(f.levels, `About ${job.item} on this page: ${f.ask}`)
+ else q[`f_${f.id}`] = jev.choice(options, `On this page, which piece of text is ${f.ask}? Pick the exact words from the page.`)
+ }
+ if (job.keep) q.keep = jev.noul(`The person wants only the ones where: ${job.keep}. Does ${job.item} on this page fit that?`)
+ return { questions: q, blocks: page.blocks.length }
+}
+
+/** Jev's answers about an item page, turned into one row. Every value came off the page. */
+export function rowFrom(page, job, answers) {
+ const row = { url: canonical(page.url), title: page.title, fields: {}, confidence: {} }
+ for (const f of job.fields) {
+ const a = answers[`f_${f.id}`]
+ if (!a) { row.fields[f.id] = ''; row.confidence[f.id] = 0; continue }
+ if (f.type === 'yesno') { row.fields[f.id] = a.noul >= 0.5 ? 'yes' : 'no'; row.confidence[f.id] = Math.max(a.noul, 1 - a.noul) }
+ else if (f.type === 'score') { const i = Math.round(Number(a.score ?? 0)); row.fields[f.id] = f.levels[Math.max(0, Math.min(f.levels.length - 1, i))] ?? ''; row.confidence[f.id] = Number(a.confidence ?? 0) }
+ else { row.fields[f.id] = blockText(page, a.choice); row.confidence[f.id] = a.choice === 'none' ? 0 : Number(a.confidence ?? 0) }
+ }
+ if (job.keep) { const k = answers.keep; row.keep = k ? k.noul >= 0.5 : true; row.keepConfidence = k ? Math.max(k.noul, 1 - k.noul) : 0 }
+ return row
+}
+
+/** The links Jev said are items, surest first, and the next-page link if it named one. */
+export function pickLinks(links, answers, { threshold = 0.5 } = {}) {
+ const items = []
+ for (const l of links) {
+ const a = answers[`l${l.n}`]
+ if (!a) continue
+ if (a.noul >= threshold) items.push({ ...l, p: a.noul })
+ }
+ items.sort((a, b) => a.n - b.n) // page order: a spreadsheet should read like the page did
+ const pick = answers.nextpage?.choice
+ const next = pick && pick !== 'none' ? links.find((l) => `l${l.n}` === pick) ?? null : null
+ return { items, next, judged: links.length }
+}
+
+/**
+ * Run a job. Every step is reported through `onEvent` so the pane can show it live.
+ * @param {object} o
+ * @param {object} o.chrome from toolchain/chrome.mjs
+ * @param {object} o.job from normalizeJob
+ * @param {function} o.ask async ({ state, questions }) => { answers, latencyMs, usage, client }
+ * @param {function} o.onEvent
+ * @param {function} [o.stopped] () => true to stop early
+ */
+export async function runJob({ chrome, job, ask, onEvent, stopped = () => false }) {
+ const out = { rows: [], pagesRead: 0, linksJudged: 0, calls: 0, errors: [], startedAt: Date.now(), finishedAt: null, stoppedEarly: false }
+ const visited = new Set()
+ const queue = [] // item pages to open
+ let listUrl = job.start
+ let listPages = 0
+
+ const say = (type, data) => { try { onEvent?.({ type, at: Date.now(), ...data }) } catch { /* the pane may be gone */ } }
+ // maxItems bounds the things collected; maxPages bounds how far down the list it walks.
+ const budgetLeft = () => out.rows.length < job.maxItems
+
+ while (listUrl && budgetLeft() && listPages < job.maxPages && !stopped()) {
+ listPages++
+ say('going', { url: listUrl, what: listPages === 1 ? 'the page you gave' : `list page ${listPages}` })
+ let page
+ try {
+ await chrome.go(listUrl)
+ page = await readPage(chrome)
+ } catch (e) { out.errors.push({ url: listUrl, message: String(e.message ?? e) }); say('trouble', { url: listUrl, message: String(e.message ?? e) }); break }
+ out.pagesRead++
+ visited.add(canonical(listUrl))
+ say('read', { url: page.url, title: page.title, links: page.links.length, blocks: page.blocks.length })
+
+ const { questions, links } = listQuestions(page, job, { wantNext: listPages < job.maxPages })
+ const res = await ask({ state: pageState(page), questions })
+ out.calls++
+ const { items, next, judged } = pickLinks(links, res.answers)
+ out.linksJudged += judged
+ say('judged', { url: page.url, kind: res.answers.kind?.choice, judged, found: items.length, latencyMs: res.latencyMs, next: next?.label ?? null, links: links.map((l) => ({ n: l.n, label: l.label, p: res.answers[`l${l.n}`]?.noul ?? 0 })).sort((a, b) => b.p - a.p).slice(0, 40) })
+
+ // The page the person pointed at may itself be the one thing they want.
+ if (listPages === 1 && res.answers.kind?.choice === 'item') {
+ const one = await collect(page)
+ if (one) { say('row', { row: one, of: out.rows.length }) }
+ }
+ for (const l of items) { const c = canonical(l.url); if (!visited.has(c)) { visited.add(c); queue.push({ ...l, url: c }) } }
+
+ // Open what was found before asking for another list page, so rows start landing at once.
+ while (queue.length && budgetLeft() && !stopped()) {
+ const link = queue.shift()
+ say('going', { url: link.url, what: link.label })
+ let itemPage
+ try {
+ await chrome.go(link.url)
+ itemPage = await readPage(chrome)
+ } catch (e) { out.errors.push({ url: link.url, message: String(e.message ?? e) }); say('trouble', { url: link.url, message: String(e.message ?? e) }); continue }
+ out.pagesRead++
+ say('read', { url: itemPage.url, title: itemPage.title, links: itemPage.links.length, blocks: itemPage.blocks.length })
+ const row = await collect(itemPage, link.label)
+ if (row) say('row', { row, of: out.rows.length })
+ }
+ listUrl = next && budgetLeft() && !stopped() ? next.url : null
+ if (!listUrl && next && budgetLeft()) out.stoppedEarly = true
+ }
+
+ async function collect(page, fromLabel = '') {
+ const { questions } = itemQuestions(page, job)
+ const res = await ask({ state: pageState(page), questions })
+ out.calls++
+ const row = rowFrom(page, job, res.answers)
+ row.n = out.rows.length + 1
+ row.fromLabel = fromLabel
+ row.latencyMs = res.latencyMs
+ row.kind = res.answers.kind?.choice ?? 'item'
+ if (row.kind === 'other' && !Object.values(row.fields).some(Boolean)) { say('skipped', { url: page.url, why: 'nothing of the job was on this page' }); return null }
+ out.rows.push(row)
+ return row
+ }
+
+ out.stoppedEarly = stopped() || !budgetLeft()
+ out.finishedAt = Date.now()
+ say('done', { rows: out.rows.length, pages: out.pagesRead, links: out.linksJudged })
+ return out
+}
diff --git a/store/agents/jev-browser/viewer/demosite.mjs b/store/agents/jev-browser/viewer/demosite.mjs
new file mode 100644
index 000000000..9c112c1bd
--- /dev/null
+++ b/store/agents/jev-browser/viewer/demosite.mjs
@@ -0,0 +1,77 @@
+// demosite.mjs — a small made-up job board, served on the loopback so the harness has something
+// real to browse the moment it opens: real HTTP, real links, real pagination, a real Chrome.
+// Only the content is invented. Point `start` at a real address and nothing else changes.
+import { createServer } from 'node:http'
+
+const COMPANIES = ['Fernhill Cloud', 'Brightpath Labs', 'Marlow & Dean', 'Kestrel Analytics', 'Northwind Health', 'Tidepool Games', 'Aster Robotics', 'Glasshouse Media']
+const TITLES = ['Senior Python Engineer', 'Data Analyst', 'Frontend Developer', 'Site Reliability Engineer', 'Product Designer', 'Machine Learning Engineer', 'Support Engineer', 'Technical Writer', 'Platform Engineer', 'QA Engineer', 'Mobile Developer', 'Security Analyst']
+const PLACES = ['London', 'Manchester', 'Remote (UK)', 'Bristol', 'Edinburgh', 'Remote (Europe)', 'Leeds', 'Cambridge']
+const TYPES = ['Permanent', 'Contract', 'Part time']
+
+const rand = (seed) => { let a = seed; return () => { a |= 0; a = (a + 0x6D2B79F5) | 0; let t = Math.imul(a ^ (a >>> 15), 1 | a); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296 } }
+
+export function makeJobs(count = 36, seed = 20260920) {
+ const r = rand(seed)
+ const pick = (l) => l[Math.floor(r() * l.length)]
+ return Array.from({ length: count }, (_, i) => {
+ const low = 30 + Math.floor(r() * 9) * 5
+ const place = pick(PLACES)
+ return {
+ id: 1000 + i, title: pick(TITLES), company: pick(COMPANIES), place,
+ salary: `£${low},000 - £${low + 10 + Math.floor(r() * 4) * 5},000`,
+ type: pick(TYPES), remote: place.startsWith('Remote'),
+ python: r() < 0.45, posted: `${1 + Math.floor(r() * 27)} September 2026`,
+ }
+ })
+}
+
+const esc = (s) => String(s).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;')
+const CSS = `body{font:15px/1.6 -apple-system,Segoe UI,Roboto,sans-serif;margin:0;background:#f6f7f9;color:#1c2330}
+header{background:#12243a;color:#fff;padding:14px 28px;display:flex;gap:22px;align-items:center}
+header a{color:#cfe0f5;text-decoration:none;font-size:14px} header b{font-size:17px}
+main{max-width:860px;margin:0 auto;padding:24px 28px 60px}
+.card{background:#fff;border:1px solid #e2e6ec;border-radius:10px;padding:16px 18px;margin-bottom:12px}
+.card h2{margin:0 0 4px;font-size:17px}.card h2 a{color:#12243a;text-decoration:none}
+.meta{color:#5b6676;font-size:13.5px}
+.pager{display:flex;gap:12px;margin-top:22px;align-items:center}
+.pager a{background:#12243a;color:#fff;padding:8px 14px;border-radius:8px;text-decoration:none;font-size:14px}
+.detail dt{color:#5b6676;font-size:13px;margin-top:12px}.detail dd{margin:2px 0 0;font-size:16px;font-weight:600}
+footer{color:#8a94a2;font-size:12.5px;padding:24px 28px;text-align:center}`
+const page = (title, body) => `<!doctype html><html lang="en"><head><meta charset="utf-8"><title>${esc(title)}</title><style>${CSS}</style></head><body>
+<header><b>Fernhill Jobs</b><a href="/">All roles</a><a href="/about">About</a><a href="/contact">Contact us</a><a href="/login">Sign in</a></header>
+<main>${body}</main><footer>A made-up job board, served on this machine for the Jev Browser harness. None of these roles exist.</footer></body></html>`
+
+/** Start the made-up site. Returns { url, close }. */
+export async function startDemoSite({ port = 0, perPage = 12, jobs = makeJobs() } = {}) {
+ const server = createServer((req, res) => {
+ const url = new URL(req.url, 'http://127.0.0.1')
+ const send = (html, code = 200) => { res.writeHead(code, { 'content-type': 'text/html; charset=utf-8', 'cache-control': 'no-store' }); res.end(html) }
+ const m = url.pathname.match(/^\/job\/(\d+)$/)
+ if (m) {
+ const job = jobs.find((j) => String(j.id) === m[1])
+ if (!job) return send(page('Not found', '<h1>No such role</h1>'), 404)
+ return send(page(`${job.title} · ${job.company}`, `<div class="card detail">
+ <h2>${esc(job.title)}</h2>
+ <p class="meta">${esc(job.company)} · ${esc(job.place)}</p>
+ <dl><dt>Salary</dt><dd>${esc(job.salary)}</dd>
+ <dt>Contract</dt><dd>${esc(job.type)}</dd>
+ <dt>Posted</dt><dd>${esc(job.posted)}</dd></dl>
+ <p>We are looking for ${esc(job.title.toLowerCase())} to join ${esc(job.company)}${job.remote ? '. This role is fully remote' : ` in our ${esc(job.place)} office`}.
+ ${job.python ? 'Day to day you will write Python and review other people\'s Python.' : 'Our stack is TypeScript end to end.'}</p>
+ <p><a href="/">Back to all roles</a></p></div>`))
+ }
+ if (url.pathname === '/about') return send(page('About', '<div class="card"><h2>About Fernhill Jobs</h2><p>A made-up job board. It exists so a browser harness has somewhere honest to practise.</p></div>'))
+ if (url.pathname === '/contact') return send(page('Contact us', '<div class="card"><h2>Contact us</h2><p>Nobody is here.</p></div>'))
+ if (url.pathname === '/login') return send(page('Sign in', '<div class="card"><h2>Sign in</h2><form><p><input name="email" placeholder="Email"></p><p><input type="password" name="password" placeholder="Password"></p><p><button type="submit">Sign in</button></p></form></div>'))
+ if (url.pathname !== '/') return send(page('Not found', '<h1>Not found</h1>'), 404)
+ const p = Math.max(1, Number(url.searchParams.get('page') || 1))
+ const slice = jobs.slice((p - 1) * perPage, p * perPage)
+ const cards = slice.map((j) => `<div class="card"><h2><a href="/job/${j.id}">${esc(j.title)}</a></h2>
+ <p class="meta">${esc(j.company)} · ${esc(j.place)} · ${esc(j.salary)}</p></div>`).join('')
+ const last = Math.ceil(jobs.length / perPage)
+ const pager = `<div class="pager">${p > 1 ? `<a href="/?page=${p - 1}">Previous page</a>` : ''}<span class="meta">Page ${p} of ${last}</span>${p < last ? `<a href="/?page=${p + 1}">Next page</a>` : ''}</div>`
+ send(page(`Fernhill Jobs · page ${p}`, `<h1>Open roles</h1><p class="meta">${jobs.length} made-up roles</p>${cards}${pager}`))
+ })
+ await new Promise((r) => server.listen(port, '127.0.0.1', r))
+ return { url: `http://127.0.0.1:${server.address().port}/`, port: server.address().port, jobs, async close() { server.closeAllConnections?.(); await new Promise((r) => server.close(r)) } }
+}
diff --git a/store/agents/jev-browser/viewer/index.html b/store/agents/jev-browser/viewer/index.html
index 374a5feb7..1ce224735 100644
--- a/store/agents/jev-browser/viewer/index.html
+++ b/store/agents/jev-browser/viewer/index.html
@@ -1,87 +1,85 @@
<!doctype html>
<html lang="en">
<head>
-<meta charset="utf-8">
-<meta name="viewport" content="width=device-width, initial-scale=1">
-<title>Jev Browser</title>
-<link rel="stylesheet" href="/base.css">
-<link rel="stylesheet" href="/studio.css">
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Jev Browser</title>
+ <link rel="icon" href="data:,">
+ <link rel="stylesheet" href="/base.css">
+ <link rel="stylesheet" href="/studio.css">
+ <script src="/jev-hud.js" defer></script>
+ <script type="module" src="/studio.js"></script>
</head>
<body>
- <div id="topbar">
- <div class="brand"><span class="dot"></span><h1 id="title">Jev Browser</h1><span class="sub">a made-up travel site · Jev picks every click</span></div>
- <div class="stats">
- <div class="stat"><label>Bookings</label><span id="s-tasks">0</span></div>
- <div class="stat"><label>Exactly right</label><span id="s-ok">—</span></div>
- <div class="stat"><label>Steps each</label><span id="s-steps">—</span></div>
- <div class="stat"><label>Lost clicks</label><span id="s-mis">0</span></div>
- <div class="stat big"><label>Seconds each</label><span id="s-secs">—</span></div>
- <div class="stat big"><label>Decisions / s</label><span id="s-rate">—</span></div>
- <div class="stat big"><label>Cost so far</label><span id="s-cost">$0</span></div>
+ <header id="topbar">
+ <div class="brand">
+ <span class="dot"></span>
+ <h1>Jev Browser</h1>
+ <span class="sub" id="taskLine">reading the job…</span>
+ <span class="pill" id="phasePill">idle</span>
+ </div>
+ <div class="stats big">
+ <div class="stat"><label>collected</label><span id="sRows">0</span></div>
+ <div class="stat"><label>pages read</label><span id="sPages">0</span></div>
+ <div class="stat"><label>links judged</label><span id="sLinks">0</span></div>
+ <div class="stat" id="rateStat"><label>decisions / s</label><span id="sRate">0</span></div>
+ <div class="stat"><label>cost so far</label><span id="sCost">$0</span></div>
</div>
<div class="controls">
- <button id="pause">Pause</button>
- <button id="step">Step</button>
- <button id="reset" class="primary">Reset</button>
+ <button id="startBtn" class="primary">Start</button>
+ <button id="stopBtn" class="hidden">Stop</button>
+ <button id="downloadBtn" title="Every row Jev collected, with its confidence">Download results.csv</button>
+ <button id="resetBtn" title="Clear the rows and read the job again">reset</button>
</div>
- </div>
+ </header>
<div id="cfgError" class="cfg-error hidden"></div>
<main>
<section id="stage">
- <div class="task panel">
- <div class="task-row">
- <span class="pill" id="taskNo">TASK 1/4</span>
- <span id="goal" class="goal"></span>
- </div>
- <div class="task-meters">
- <span><b id="m-time">0.0</b> s</span><span><b id="m-steps">0</b> steps</span><span><b id="m-cost">$0</b></span>
- <span id="m-note" class="note"></span>
- </div>
+ <div id="addressBar" class="address">
+ <span class="lock" id="lockIcon" title="Only http and https, and only the sites this job names">⛨</span>
+ <input id="urlInput" type="text" spellcheck="false" autocomplete="off" placeholder="the browser is not open yet">
+ <button id="goBtn" title="Send the browser to this address">Go</button>
</div>
- <div class="window" id="window">
- <div class="chrome">
- <span class="lights"><i></i><i></i><i></i></span>
- <span class="tab"><span class="fav"></span><span id="tabTitle">SkyHop</span></span>
- <span class="url"><span class="lock">●</span><span id="url">https://skyhop.example/</span></span>
- </div>
- <div class="viewport" id="viewport">
- <div class="page" id="page"></div>
- <div class="cursor" id="cursor"><svg viewBox="0 0 24 24" width="26" height="26"><path d="M4 2l15 9-7 1.6L9.6 20z" fill="#fff" stroke="#0b0d14" stroke-width="1.6" stroke-linejoin="round"/></svg></div>
- <div class="stamp hidden" id="stamp"></div>
+ <div id="screenWrap" class="screen">
+ <img id="screen" alt="">
+ <div id="screenIdle" class="screen-idle">
+ <b>A real Chrome, driven by Jev</b>
+ <span id="screenIdleSub">Press Start. A browser window opens and Jev reads real pages: one call per page judges every link on it and pulls every field at once.</span>
+ <div class="screen-note" id="chromeNote"></div>
</div>
</div>
- <div class="toolbar panel">
- <label class="slider">Distraction <input id="distraction" type="range" min="0" max="1" step="0.05" value="0.25"> <b id="distractionVal">0.25</b></label>
- <label class="slider">Step <input id="stepMs" type="range" min="60" max="900" step="10" value="170"> <b id="stepMsVal">170</b> ms</label>
- <button id="popup">Throw a pop-up</button>
- <button id="nextTask">Next task</button>
- <button id="heat" class="on">Show Jev's mind</button>
- <span class="hint">The page is live: <b>click anything on it yourself</b> and Jev has to deal with what you did.</span>
+ <div class="results-head">
+ <b>Collected</b>
+ <span class="dim" id="resultsNote">nothing yet</span>
+ <span class="right dim">every value is text taken off the page, never written by Jev</span>
</div>
+ <div class="table-wrap"><table id="results"><thead><tr id="resultsHead"></tr></thead><tbody id="resultsBody"></tbody></table></div>
</section>
<aside id="rail">
- <div class="panel">
- <div class="panel-title">What Jev reads</div>
- <pre id="stateText" class="statetext"></pre>
- </div>
- <div class="panel">
- <div class="panel-title">How it works</div>
- <div class="how">
- Jev never sees the screen. Each step the page is written out as the text above, and Jev
- picks <b>one element out of the list</b>, a new list every step, plus two yes/no reads, in one
- call. Clicking an input fills it with the right value from the task.<br><br>
- <b>Distraction</b> adds cookie walls, pop-ups, decoy buttons and <b>layout shifts</b> that land
- between reading the page and clicking it, so a click can hit the wrong thing. Every
- booking is checked against the task. The site and the flights are made up.
+ <section class="panel" id="jobPanel">
+ <div class="panel-title">The job <span class="right dim" id="jobSource">browse.json</span></div>
+ <div id="jobBody" class="job-body"></div>
+ </section>
+ <section class="panel">
+ <div class="panel-title">Jev on this page's links <span class="right dim" id="linkNote">one call, one answer each</span></div>
+ <div id="linkList" class="link-list"><div class="dim small">Nothing judged yet.</div></div>
+ </section>
+ <div data-jev-hud></div>
+ <section class="panel">
+ <div class="panel-title">What it did</div>
+ <div id="feed" class="feed"></div>
+ </section>
+ <section class="panel">
+ <div class="panel-title">What it will not do</div>
+ <div class="small dim guard">
+ It only reads. It never submits a form, never presses anything that reads like pay, buy,
+ delete or send, and never types into a password, card or code field. It stays on the sites
+ the job names. The window is yours: sign in there yourself, and it keeps you signed in next
+ time.
</div>
- </div>
- <div class="panel">
- <div class="panel-title">Bookings</div>
- <ul id="feed"></ul>
- </div>
+ </section>
</aside>
</main>
-<script src="/studio.js"></script>
-<script src="/jev-hud.js" defer></script>
+ <div id="toast" class="toast hidden"></div>
</body>
</html>
diff --git a/store/agents/jev-browser/viewer/mock.mjs b/store/agents/jev-browser/viewer/mock.mjs
index 210d7b56b..d1e83beae 100644
--- a/store/agents/jev-browser/viewer/mock.mjs
+++ b/store/agents/jev-browser/viewer/mock.mjs
@@ -1,82 +1,93 @@
-// mock.mjs — the offline stand-in for Jev in this harness. It reads ONLY the state text and the
-// option descriptions that live Jev also gets, and answers with a full probability distribution.
-// It is a careful form-filler, not Jev's judgement; the pane badges it MOCK.
+// mock.mjs — the offline stand-in, so the harness runs with no key.
+//
+// It reads the same words Jev is shown and answers with crude rules: a link that looks like
+// navigation is not an item, a field is whatever nearby text shares the most words with the ask.
+// It exercises the plumbing. It is NOT Jev's judgement, and the pane badges it MOCK.
+import { stems, hash01 } from '../toolchain/jev.mjs'
-export function readTask(text) {
- const m = text.match(/TASK: Book the (cheapest|earliest|latest) (nonstop )?flight from (\S+) to (\S+) on (\S+) for (.+?) \((\S+?)\), (no|\d+) checked bags?/)
- if (!m) return null
- return { pick: m[1], nonstop: !!m[2], from: m[3], to: m[4], day: m[5], name: m[6], email: m[7], bags: m[8] === 'no' ? 0 : Number(m[8]) }
+const NAV = /\b(home|about|contact|sign in|sign up|log ?in|log ?out|register|privacy|terms|cookies?|help|support|faq|careers?|blog|news|press|search|menu|skip to|back to|previous|next page|page \d+|all roles|share|tweet|facebook|linkedin)\b/i
+const NEXT = /\b(next|more|older|page \d+|›|»|>)\b/i
+const MONEY = /[£$€]\s?\d|\d[\d,.]*\s?(k\b|,000|per (year|annum|hour|day))/i
+const DATE = /\b(\d{1,2} )?(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)[a-z]* \d{1,4}\b|\b\d{4}-\d{2}-\d{2}\b|\b\d+ (day|week|month|hour)s? ago\b/i
+
+const overlap = (want, have) => {
+ const a = new Set(stems(want)), b = stems(have)
+ if (!a.size || !b.length) return 0
+ let hit = 0
+ for (const w of b) if (a.has(w)) hit++
+ return hit / Math.sqrt(a.size)
}
-const toMin = (s) => { const [h, m] = s.split(':').map(Number); return h * 60 + m }
+/**
+ * @param {() => ({ expecting?: 'list'|'item', item?: string })} context what the viewer is looking at
+ */
+export function browserMock(context = () => ({})) {
+ return (stateText, qid, q, salt) => {
+ const ctx = context() ?? {}
+ const jitter = (k) => (hash01(`${qid}:${k}:${stateText.length}`, salt) - 0.5) * 0.12
-function parseRow(desc) {
- const m = desc.match(/(Sponsored · )?(.+?) · (\S+) to (\S+) · departs (\d\d:\d\d) arrives (\d\d:\d\d) · (nonstop|\d+ stops?) · \$(\d+)/)
- if (!m) return null
- return { sponsored: !!m[1], from: m[3], to: m[4], dep: toMin(m[5]), stops: m[7] === 'nonstop' ? 0 : parseInt(m[7], 10), price: Number(m[8]) }
-}
+ // What kind of page: the viewer already knows why it opened this one.
+ if (qid === 'kind' && q.type === 'choice') {
+ const best = ctx.expecting === 'item' ? 'item' : ctx.expecting === 'list' ? 'list' : 'other'
+ const probabilities = {}
+ for (const o of q.options) probabilities[o] = o === best ? 0.82 : 0.09
+ return { type: 'choice', choice: best, confidence: 0.82, probabilities }
+ }
-/** Which element a careful person would act on next, from the text alone. */
-export function nextTarget(text, options, descriptions) {
- const task = readTask(text)
- if (!task) return null
- const has = (id) => options.includes(id)
- const valueOf = (id) => (descriptions[id]?.match(/value "([^"]*)"/) ?? [])[1]
- const page = (text.match(/^PAGE: (\w+)/m) ?? [])[1]
+ // Is this link one of the things? The instruction carries the link's words and its path.
+ if (/^l\d+$/.test(qid) && q.type === 'noul') {
+ const m = q.instructions.match(/^"(.*?)" is the text of a link on this page, pointing at (\S+?)\./)
+ const label = m?.[1] ?? '', path = m?.[2] ?? ''
+ if (!label) return { type: 'noul', noul: 0.1 }
+ let p = 0.28
+ if (NAV.test(label)) p -= 0.22
+ if (/\/\d{2,}(\/|$)|\/[a-z0-9-]{12,}(\/|$)/i.test(path)) p += 0.4 // an id or a long slug: a thing, not a section
+ if (path.split('/').filter(Boolean).length >= 2) p += 0.1
+ if (label.length > 14) p += 0.12
+ if (NEXT.test(label)) p -= 0.3
+ p += overlap(ctx.item ?? '', label) * 0.25
+ return { type: 'noul', noul: Math.max(0.02, Math.min(0.97, p + jitter(label))) }
+ }
- if (/^OVERLAY: a /m.test(text)) return ['cookie-reject', 'popup-no', 'popup-close', 'cookie-accept'].find(has) ?? null
- if (page === 'promo') return has('back-to-booking') ? 'back-to-booking' : null
- if (page === 'search') {
- for (const id of ['from', 'to', 'day']) if (has(id) && valueOf(id) === '') return id
- return has('search-flights') ? 'search-flights' : null
- }
- if (page === 'results') {
- const rows = options.filter((id) => id.startsWith('select-')).map((id) => ({ id, ...parseRow(descriptions[id] ?? '') })).filter((r) => r.to)
- const ok = rows.filter((r) => !r.sponsored && r.to === task.to && r.from === task.from && (!task.nonstop || r.stops === 0))
- if (!ok.length) return task.nonstop && has('nonstop-only') && valueOf('nonstop-only') === 'on' ? 'nonstop-only' : null
- const by = task.pick === 'cheapest' ? (r) => r.price : task.pick === 'earliest' ? (r) => r.dep : (r) => -r.dep
- return ok.sort((a, b) => by(a) - by(b))[0].id
- }
- if (page === 'details') {
- const sum = text.match(/· (\S+) to (\S+) · departs/)
- if (sum && sum[2] !== task.to && has('change-flight')) return 'change-flight'
- if (task.nonstop && /· \d+ stops? · \$/.test(text.match(/^ {2}.*departs.*$/m)?.[0] ?? '') && has('change-flight')) return 'change-flight'
- if (has('name') && valueOf('name') === '') return 'name'
- if (has('email') && valueOf('email') === '') return 'email'
- const bags = Number((text.match(/Checked bags: (\d+)/) ?? [])[1] ?? 0)
- if (bags < task.bags && has('bag-add')) return 'bag-add'
- if (bags > task.bags && has('bag-remove')) return 'bag-remove'
- if (has('insurance') && valueOf('insurance') === 'on') return 'insurance'
- return has('continue') ? 'continue' : null
- }
- if (page === 'review') {
- if (has('flex') && valueOf('flex') === 'on') return 'flex'
- const who = text.match(/· (\d+) checked bag\(s\)( · insurance)?/)
- if (who && (Number(who[1]) !== task.bags || who[2])) return 'back'
- return has('confirm') ? 'confirm' : null
- }
- return null
-}
+ // Which link is the next page.
+ if (qid === 'nextpage' && q.type === 'choice') {
+ const probabilities = {}
+ let best = 'none', bestScore = 0.25
+ for (const o of q.options) {
+ const text = `${o} ${q.descriptions?.[o] ?? ''}`
+ const s = o === 'none' ? 0.2 : /next/i.test(text) ? 0.95 : NEXT.test(text) ? 0.6 : 0.05
+ probabilities[o] = s
+ if (s > bestScore) { best = o; bestScore = s }
+ }
+ const total = Object.values(probabilities).reduce((a, b) => a + b, 0) || 1
+ for (const k of Object.keys(probabilities)) probabilities[k] /= total
+ return { type: 'choice', choice: best, confidence: Math.min(0.95, probabilities[best] * 2), probabilities }
+ }
+
+ // A field: pick the piece of page text that shares the most words with what was asked for.
+ if (qid.startsWith('f_') && q.type === 'choice') {
+ const ask = q.instructions.replace(/^On this page, which piece of text is /, '').replace(/\?.*$/, '')
+ const wantsMoney = /salary|pay|price|cost|wage|fee|amount/i.test(ask)
+ const wantsDate = /date|posted|published|when|deadline/i.test(ask)
+ let best = 'none', bestScore = 0.18
+ const scores = {}
+ for (const o of q.options) {
+ if (o === 'none') { scores[o] = 0.18; continue }
+ const text = q.descriptions?.[o] ?? ''
+ let s = overlap(ask, text) * 0.8
+ if (wantsMoney && MONEY.test(text)) s += 0.75
+ if (wantsDate && DATE.test(text)) s += 0.7
+ if (text.length > 120) s -= 0.25
+ s += jitter(o) * 0.5
+ scores[o] = Math.max(0, s)
+ if (s > bestScore) { best = o; bestScore = s }
+ }
+ const total = Object.values(scores).reduce((a, b) => a + b, 0) || 1
+ const probabilities = {}
+ for (const [k, v] of Object.entries(scores)) probabilities[k] = v / total
+ return { type: 'choice', choice: best, confidence: Math.max(0.2, Math.min(0.96, bestScore)), probabilities }
+ }
-export function browserMock(text, id, q) {
- if (!/^TASK: /m.test(text)) return null // not our state text: let the generic mock answer
- if (id === 'action') {
- const options = q.options ?? []
- if (!options.length) return null
- const target = nextTarget(text, options, q.descriptions ?? {})
- const words = new Set((text.match(/^TASK: .*$/m)?.[0] ?? '').toLowerCase().split(/[^a-z0-9]+/).filter((w) => w.length > 2))
- // Everything that is not the target still gets a little mass, more if it shares the task's words.
- const raw = options.map((o) => {
- if (o === target) return 5.2
- const d = `${o} ${q.descriptions?.[o] ?? ''}`.toLowerCase().split(/[^a-z0-9]+/)
- return 0.6 + 0.5 * d.filter((w) => words.has(w)).length
- })
- const exps = raw.map((r) => Math.exp(r - Math.max(...raw))), sum = exps.reduce((a, b) => a + b, 0)
- const probabilities = Object.fromEntries(options.map((o, i) => [o, exps[i] / sum]))
- const choice = target ?? options[exps.indexOf(Math.max(...exps))]
- return { type: 'choice', choice, confidence: probabilities[choice], probabilities }
+ return null // everything else: the generic reader in jev.mjs
}
- if (id === 'done') return { type: 'noul', noul: /^PAGE: done/m.test(text) ? 0.97 : 0.02 }
- if (id === 'obstacle') return { type: 'noul', noul: /^OVERLAY: a /m.test(text) ? 0.95 : 0.04 }
- return null
}
diff --git a/store/agents/jev-browser/viewer/page.mjs b/store/agents/jev-browser/viewer/page.mjs
new file mode 100644
index 000000000..04a055734
--- /dev/null
+++ b/store/agents/jev-browser/viewer/page.mjs
@@ -0,0 +1,169 @@
+// page.mjs — turn a live web page into something Jev can answer about.
+//
+// Jev never writes text. So every value this harness can ever put in a cell is a piece of text that
+// is really on the page: the reader numbers the page's links and its short text blocks, and Jev
+// picks one by number. A made-up value is not possible, because there is nothing to make it out of.
+//
+// The reader runs inside the page. It marks what it found with data-jev-n, so a later click or read
+// points at exactly the element that was offered.
+
+/** The reader, as source, so it can be handed to Runtime.evaluate. */
+export const READER = `(() => {
+ const MAX_LINKS = 220, MAX_BLOCKS = 90, MAX_LABEL = 110, MAX_BLOCK = 180
+ for (const e of document.querySelectorAll('[data-jev-n]')) e.removeAttribute('data-jev-n')
+ const seen = document.documentElement.getBoundingClientRect()
+ const vis = (e) => {
+ const r = e.getBoundingClientRect()
+ if (r.width < 4 || r.height < 4) return null
+ if (r.bottom < -400 || r.top > seen.height + 400) return null
+ const s = getComputedStyle(e)
+ if (s.visibility === 'hidden' || s.display === 'none' || Number(s.opacity) < 0.05) return null
+ return { x: Math.round(r.x), y: Math.round(r.y), w: Math.round(r.width), h: Math.round(r.height) }
+ }
+ const tidy = (s, max) => String(s || '').replace(/\\s+/g, ' ').trim().slice(0, max)
+ const labelOf = (e) => {
+ const own = tidy(e.innerText || e.textContent, MAX_LABEL)
+ if (own) return own
+ const img = e.querySelector('img[alt]')
+ return tidy(e.getAttribute('aria-label') || e.getAttribute('title') || e.getAttribute('value') || e.getAttribute('placeholder') || (img && img.getAttribute('alt')) || '', MAX_LABEL)
+ }
+ let n = 0
+ const mark = (e) => { const id = ++n; e.setAttribute('data-jev-n', String(id)); return id }
+
+ // ---- links: where the page can go next -------------------------------------------------------
+ const links = [], byHref = new Map()
+ for (const a of document.querySelectorAll('a[href]')) {
+ if (links.length >= MAX_LINKS) break
+ let href
+ try { href = new URL(a.getAttribute('href'), location.href) } catch { continue }
+ if (href.protocol !== 'http:' && href.protocol !== 'https:') continue
+ href.hash = ''
+ const url = href.toString()
+ if (url === location.href.split('#')[0]) continue
+ const box = vis(a)
+ if (!box) continue
+ const label = labelOf(a)
+ if (!label) continue
+ const had = byHref.get(url)
+ if (had) { if (label.length > had.label.length) had.label = label; continue } // one row per address
+ const row = { n: mark(a), label, url, path: href.pathname + (href.search || ''), box }
+ byHref.set(url, row)
+ links.push(row)
+ }
+
+ // ---- controls: what a person could press ------------------------------------------------------
+ const controls = []
+ for (const e of document.querySelectorAll('button, input, select, textarea, [role="button"], [role="tab"], [role="checkbox"]')) {
+ if (controls.length >= 60) break
+ const box = vis(e)
+ if (!box) continue
+ const tag = e.tagName.toLowerCase()
+ const type = (e.getAttribute('type') || '').toLowerCase()
+ const label = labelOf(e) || tidy(e.name || e.id, MAX_LABEL)
+ if (!label) continue
+ controls.push({ n: mark(e), label, tag, type, inForm: !!e.closest('form'), box })
+ }
+
+ // ---- blocks: the text a value could be --------------------------------------------------------
+ // Leaf-ish elements only, so "£45,000" is offered and not the whole page wrapped around it.
+ // Whatever is in the page's own furniture is not one of its values, so headers, navigation and
+ // footers are left out. A line that mashes values together ("Acme Ltd · London · £45,000") is
+ // also offered in parts, because a person wants the company in the company column.
+ const FURNITURE = 'header, footer, nav, aside, [role="banner"], [role="contentinfo"], [role="navigation"], [role="search"]'
+ const LABELLED = /^(dt|th)$/i // a label whose value sits in the element next to it
+ const SPLIT = /\\s+[·|•\\u2014\\u2013\\u2022]\\s+/
+ const blocks = [], sawText = new Set(), consumed = new Set()
+ const ownText = (e) => tidy(Array.from(e.childNodes).filter((c) => c.nodeType === 3).map((c) => c.textContent).join(' '), MAX_BLOCK)
+ // Numbers a person would want on their own: "In stock (19 available)" also offers "19".
+ const numbersIn = (t) => (t.match(/\\d[\\d,.]*/g) || []).map((x) => x.replace(/[.,]$/, '')).filter((x) => x.length && x !== t).slice(0, 2)
+ const walker = document.createTreeWalker(document.body || document.documentElement, NodeFilter.SHOW_ELEMENT)
+ for (let e = walker.currentNode; e && blocks.length < MAX_BLOCKS; e = walker.nextNode()) {
+ if (/^(script|style|noscript|svg|head|meta|link)$/i.test(e.tagName)) continue
+ if (consumed.has(e)) continue
+ let text = ownText(e)
+ let label = ''
+ // A label and its value belong together, or the value gets dropped as a repeat of one further up
+ // and the label is left looking like an empty promise.
+ if (LABELLED.test(e.tagName)) {
+ const next = e.nextElementSibling
+ const value = next && /^(dd|td)$/i.test(next.tagName) ? ownText(next) : ''
+ if (text && value) { consumed.add(next); label = text; text = tidy(text.replace(/:\\s*$/, '') + ': ' + value, MAX_BLOCK) }
+ }
+ if (text.length < 2) continue
+ if (!vis(e)) continue
+ if (e.closest(FURNITURE)) continue
+ const key = text.toLowerCase()
+ if (sawText.has(key)) continue
+ sawText.add(key)
+ let parts = SPLIT.test(text) ? text.split(SPLIT).map((p) => p.trim()) : []
+ if (label) parts = [text.slice(label.replace(/:\\s*$/, '').length + 2)]
+ parts = parts.map((p) => p.trim()).filter((p) => p.length > 1 && p.length < 80)
+ for (const p of [...parts, text]) for (const num of numbersIn(p)) if (!parts.includes(num)) parts.push(num)
+ blocks.push({ n: mark(e), text, tag: e.tagName.toLowerCase(), parts: parts.slice(0, 5) })
+ }
+
+ const heads = Array.from(document.querySelectorAll('h1, h2')).map((h) => tidy(h.innerText, 120)).filter(Boolean).slice(0, 6)
+ return {
+ url: location.href, title: tidy(document.title, 160), heads,
+ text: tidy(document.body ? document.body.innerText : '', 4000),
+ links, controls, blocks,
+ counts: { links: document.querySelectorAll('a[href]').length, blocks: blocks.length },
+ }
+})()`
+
+/** Read the live page. Returns the shape above, with `path` selectors ready for a click. */
+export async function readPage(chrome) {
+ const page = await chrome.evaluate(READER)
+ if (!page) throw new Error('the page could not be read')
+ for (const list of [page.links, page.controls, page.blocks]) for (const row of list) row.css = `[data-jev-n="${row.n}"]`
+ return page
+}
+
+/** What Jev is shown about the page: short, and the same text a person would see. */
+export function pageState(page, extra = {}) {
+ return {
+ page_title: page.title,
+ page_address: page.url,
+ headings: page.heads.join(' | '),
+ page_text: page.text.slice(0, 2500),
+ ...extra,
+ }
+}
+
+/** The numbered text blocks, as choice options: the key is the number, the meaning is the text. */
+export const MAX_OPTIONS = 250 // a choice takes at most 255
+export function blockOptions(page, { none = 'nothing on this page says it' } = {}) {
+ const options = { none }
+ let n = 1
+ for (const b of page.blocks) {
+ if (n >= MAX_OPTIONS) break
+ options[`b${b.n}`] = b.text
+ n++
+ // A line that holds more than one value is offered whole and in parts, so a column can take the
+ // part it wants: the company out of "Acme · Leeds", the 19 out of "In stock (19 available)".
+ b.parts?.forEach((p, i) => { if (n < MAX_OPTIONS) { options[`b${b.n}p${i}`] = p; n++ } })
+ }
+ return options
+}
+
+/** Turn Jev's pick back into the text that was on the page. */
+export function blockText(page, pick) {
+ if (!pick || pick === 'none') return ''
+ const m = String(pick).match(/^b(\d+)(?:p(\d+))?$/)
+ if (!m) return ''
+ const block = page.blocks.find((b) => b.n === Number(m[1]))
+ if (!block) return ''
+ return m[2] === undefined ? block.text : block.parts?.[Number(m[2])] ?? block.text
+}
+
+const CLEAN = /(^|[?&])(utm_[^=]+|fbclid|gclid|ref|source)=[^&]*/gi
+/** One address, tidied, so the same page is not visited twice under two names. */
+export function canonical(url) {
+ try {
+ const u = new URL(url)
+ u.hash = ''
+ u.search = u.search.replace(CLEAN, '$1').replace(/[?&]+$/, '').replace(/&&+/g, '&').replace(/\?&/, '?')
+ if (u.pathname.length > 1 && u.pathname.endsWith('/')) u.pathname = u.pathname.slice(0, -1)
+ return u.toString()
+ } catch { return String(url) }
+}
diff --git a/store/agents/jev-browser/viewer/sim.mjs b/store/agents/jev-browser/viewer/sim.mjs
deleted file mode 100644
index ba9a7ddc6..000000000
--- a/store/agents/jev-browser/viewer/sim.mjs
+++ /dev/null
@@ -1,289 +0,0 @@
-// sim.mjs — a tiny made-up travel site ("SkyHop") and the booking tasks Jev runs on it.
-// The site is a five-page funnel (search, results, details, review, done) rendered to a flat list of
-// elements with rectangles on a 1000x620 page, so the same data drives what Jev reads, where the
-// cursor goes and what a late click lands on. Pure functions over a plain `world` object.
-import { mulberry32 } from './kit.mjs'
-
-export const PAGE_W = 1000, PAGE_H = 620
-export const DEFAULT = {
- title: 'Jev Browser',
- description: 'Jev books flights on a made-up travel site, one click at a time.',
- site: 'SkyHop',
- stepMs: 170, // one Jev decision (one click or one field) per step
- distraction: 0.25, // 0..1 — cookie walls, pop-ups, decoy buttons and layout shifts. THE DIAL.
- flights: 7, // result rows per search (4..9)
- maxSteps: 45, // a task fails if it takes more steps than this
- seed: 11,
- style: 'Finish the task in as few actions as you can. Fill every field the task gives you, choose exactly the flight it asks for, refuse every extra, and close anything that gets in the way.',
- tasks: [
- { from: 'SFO', to: 'JFK', day: 'Friday', pick: 'cheapest', nonstop: true, name: 'Ada Park', email: '[email protected]', bags: 1 },
- { from: 'SEA', to: 'AUS', day: 'Monday', pick: 'earliest', nonstop: false, name: 'Omar Reyes', email: '[email protected]', bags: 0 },
- { from: 'BOS', to: 'LAX', day: 'Sunday', pick: 'cheapest', nonstop: false, name: 'Mei Tanaka', email: '[email protected]', bags: 2 },
- { from: 'DEN', to: 'MIA', day: 'Wednesday', pick: 'latest', nonstop: true, name: 'Lena Okafor', email: '[email protected]', bags: 1 },
- ],
-}
-
-const AIRLINES = ['Nimbus', 'Kestrel', 'Blue Heron', 'Polar', 'Sundial', 'Alto']
-const NEAR = { JFK: 'EWR', LAX: 'BUR', MIA: 'FLL', AUS: 'SAT', SFO: 'OAK', SEA: 'PDX', BOS: 'PVD', DEN: 'COS' }
-const pad = (n) => String(n).padStart(2, '0')
-const hhmm = (m) => `${pad(Math.floor(m / 60) % 24)}:${pad(m % 60)}`
-
-export function goalText(t) {
- const bags = t.bags === 0 ? 'no checked bags' : t.bags === 1 ? '1 checked bag' : `${t.bags} checked bags`
- return `Book the ${t.pick} ${t.nonstop ? 'nonstop ' : ''}flight from ${t.from} to ${t.to} on ${t.day} for ${t.name} (${t.email}), ${bags}. No extras.`
-}
-
-export function cleanTask(t, i = 0) {
- const s = (v, d) => (typeof v === 'string' && v.trim() ? v.trim().slice(0, 40) : d)
- const pick = ['cheapest', 'earliest', 'latest'].includes(t?.pick) ? t.pick : 'cheapest'
- return { from: s(t?.from, 'SFO').toUpperCase().slice(0, 4), to: s(t?.to, 'JFK').toUpperCase().slice(0, 4), day: s(t?.day, 'Friday'), pick, nonstop: !!t?.nonstop, name: s(t?.name, `Guest ${i + 1}`).replace(/[()]/g, ''), email: s(t?.email, `guest${i + 1}@example.com`).replace(/[\s()]/g, ''), bags: Math.max(0, Math.min(3, Math.round(Number(t?.bags) || 0))) }
-}
-
-function makeFlights(task, n, rng) {
- const rows = []
- for (let i = 0; i < n; i++) {
- const dep = 5 * 60 + Math.floor(rng() * 16 * 12) * 5, stops = rng() < 0.45 ? 0 : rng() < 0.75 ? 1 : 2
- rows.push({ id: `f${i + 1}`, airline: AIRLINES[(rng() * AIRLINES.length) | 0], dep, arr: dep + 170 + stops * 95 + Math.floor(rng() * 12) * 5, stops, price: 120 + Math.floor(rng() * 56) * 5 + (stops === 0 ? 60 : 0), to: task.to, sponsored: false })
- }
- if (task.nonstop && !rows.some((r) => r.stops === 0)) rows[(rng() * rows.length) | 0].stops = 0
- // No ties on the thing the task sorts by, so there is exactly one right answer.
- const key = task.pick === 'cheapest' ? 'price' : 'dep'
- const seen = new Set()
- for (const r of rows) { while (seen.has(r[key])) r[key] += 5; seen.add(r[key]) }
- rows.sort((a, b) => a.dep - b.dep)
- return rows
-}
-
-export function bestFlight(task, flights) {
- const ok = flights.filter((f) => !f.sponsored && f.to === task.to && (!task.nonstop || f.stops === 0))
- const by = task.pick === 'cheapest' ? (f) => f.price : task.pick === 'earliest' ? (f) => f.dep : (f) => -f.dep
- return ok.sort((a, b) => by(a) - by(b))[0] ?? null
-}
-
-export function createWorld(cfg, taskIndex = 0, round = 0) {
- const tasks = (Array.isArray(cfg.tasks) && cfg.tasks.length ? cfg.tasks : DEFAULT.tasks).map(cleanTask)
- const task = tasks[taskIndex % tasks.length]
- const rng = mulberry32((Number(cfg.seed) || 11) + taskIndex * 131 + round * 7919)
- const flights = makeFlights(task, Math.max(4, Math.min(9, Math.round(cfg.flights) || 7)), rng)
- const d = cfg.distraction
- return {
- rng, task, taskIndex, taskCount: tasks.length, goal: goalText(task),
- page: 'search', flights, best: bestFlight(task, flights),
- form: { from: '', to: '', day: '', promo: '' }, nonstopOnly: false, sort: 'dep', sponsoredShown: false,
- chosen: null, details: { name: '', email: '', bags: 0, insurance: false, newsletter: false }, flex: false,
- overlay: rng() < d * 0.9 ? { kind: 'cookie' } : null,
- steps: 0, wasted: 0, misclicks: 0, startedAt: null, finishedAt: null,
- status: 'running', result: null, history: [], lastAction: null, shake: null,
- }
-}
-
-// ---------------------------------------------------------------- layout: page -> elements
-const el = (id, role, label, rect, extra = {}) => ({ id, role, label, rect, act: role !== 'text' && role !== 'card', ...extra })
-
-function rowText(f, task) {
- const stops = f.stops === 0 ? 'nonstop' : f.stops === 1 ? '1 stop' : `${f.stops} stops`
- return `${f.sponsored ? 'Sponsored · ' : ''}${f.airline} · ${task.from} to ${f.to} · departs ${hhmm(f.dep)} arrives ${hhmm(f.arr)} · ${stops} · $${f.price}`
-}
-
-export function visibleFlights(world) {
- let rows = world.flights.filter((f) => !world.nonstopOnly || f.stops === 0)
- rows = [...rows].sort((a, b) => (world.sort === 'price' ? a.price - b.price : world.sort === 'late' ? b.dep - a.dep : a.dep - b.dep))
- if (world.sponsoredShown) rows = [world.sponsoredRow, ...rows]
- return rows.slice(0, 8)
-}
-
-export function elements(world, cfg) {
- const d = cfg.distraction, t = world.task, out = []
- out.push(el('logo', 'text', cfg.site, [28, 14, 160, 30], { tone: 'logo' }))
- if (d > 0.15) for (const [i, name] of ['Hotels', 'Cars', 'Deals'].entries()) out.push(el(`nav-${name.toLowerCase()}`, 'link', name, [760 + i * 78, 14, 70, 30], { tone: 'decoy' }))
-
- if (world.page === 'search') {
- out.push(el('h', 'text', 'Where to?', [60, 84, 400, 44], { tone: 'h1' }))
- out.push(el('card', 'card', '', [40, 140, 920, d > 0.3 ? 300 : 230]))
- out.push(el('from', 'input', 'From', [70, 180, 250, 56], { value: world.form.from }))
- out.push(el('to', 'input', 'To', [340, 180, 250, 56], { value: world.form.to }))
- out.push(el('day', 'input', 'Day', [610, 180, 200, 56], { value: world.form.day }))
- const ready = world.form.from && world.form.to && world.form.day
- out.push(el('search-flights', 'button', 'Search flights', [70, 270, 220, 54], { tone: 'primary', disabled: !ready, note: ready ? '' : 'disabled until From, To and Day are filled' }))
- if (d > 0.1) out.push(el('search-hotels', 'button', 'Search hotels', [310, 270, 190, 54], { tone: 'decoy' }))
- if (d > 0.45) out.push(el('search-cars', 'button', 'Search flights + cars', [520, 270, 230, 54], { tone: 'decoy' }))
- if (d > 0.3) out.push(el('promo', 'input', 'Promo code (optional)', [70, 356, 300, 50], { value: world.form.promo, tone: 'decoy' }))
- } else if (world.page === 'results') {
- out.push(el('h', 'text', `${t.from} to ${t.to} · ${t.day}`, [60, 74, 500, 40], { tone: 'h1' }))
- out.push(el('nonstop-only', 'toggle', 'Nonstop only', [60, 122, 170, 38], { value: world.nonstopOnly ? 'on' : 'off' }))
- out.push(el('sort-price', 'button', 'Cheapest first', [250, 122, 160, 38], { tone: world.sort === 'price' ? 'on' : 'quiet' }))
- out.push(el('sort-early', 'button', 'Earliest first', [420, 122, 160, 38], { tone: world.sort === 'dep' ? 'on' : 'quiet' }))
- out.push(el('sort-late', 'button', 'Latest first', [590, 122, 150, 38], { tone: world.sort === 'late' ? 'on' : 'quiet' }))
- visibleFlights(world).forEach((f, i) => {
- const y = 176 + i * 54
- out.push(el(`row-${f.id}`, 'text', rowText(f, t), [60, y, 730, 46], { tone: f.sponsored ? 'sponsored' : 'row' }))
- out.push(el(`select-${f.id}`, 'button', 'Select', [806, y + 3, 134, 40], { tone: f.sponsored ? 'decoy' : 'primary', context: rowText(f, t) }))
- })
- } else if (world.page === 'details') {
- const f = world.chosen
- out.push(el('h', 'text', 'Passenger details', [60, 74, 500, 40], { tone: 'h1' }))
- out.push(el('sum', 'text', f ? rowText(f, t) : '', [60, 118, 880, 34], { tone: 'row' }))
- out.push(el('name', 'input', 'Full name', [60, 176, 400, 56], { value: world.details.name }))
- out.push(el('email', 'input', 'Email', [480, 176, 400, 56], { value: world.details.email }))
- out.push(el('bags', 'text', `Checked bags: ${world.details.bags}`, [60, 262, 220, 44], { tone: 'label' }))
- out.push(el('bag-remove', 'button', 'Remove a bag', [290, 262, 160, 44], { tone: 'quiet', disabled: world.details.bags === 0 }))
- out.push(el('bag-add', 'button', 'Add a bag', [462, 262, 140, 44], { tone: 'quiet', disabled: world.details.bags >= 3 }))
- if (d > 0.1) out.push(el('insurance', 'toggle', 'Add travel insurance +$49', [60, 334, 310, 44], { value: world.details.insurance ? 'on' : 'off', tone: 'decoy' }))
- if (d > 0.35) out.push(el('newsletter', 'toggle', 'Email me deals', [390, 334, 200, 44], { value: world.details.newsletter ? 'on' : 'off', tone: 'decoy' }))
- const ready = world.details.name && world.details.email
- out.push(el('continue', 'button', 'Continue', [60, 414, 200, 54], { tone: 'primary', disabled: !ready, note: ready ? '' : 'disabled until name and email are filled' }))
- out.push(el('change-flight', 'button', 'Change flight', [280, 414, 180, 54], { tone: 'quiet' }))
- } else if (world.page === 'review') {
- const f = world.chosen
- out.push(el('h', 'text', 'Review your booking', [60, 74, 500, 40], { tone: 'h1' }))
- out.push(el('sum', 'text', f ? rowText(f, t) : '', [60, 124, 880, 34], { tone: 'row' }))
- out.push(el('who', 'text', `${world.details.name} · ${world.details.email} · ${world.details.bags} checked bag(s)${world.details.insurance ? ' · insurance +$49' : ''}${world.flex ? ' · Flex +$80' : ''}`, [60, 166, 880, 34], { tone: 'row' }))
- out.push(el('total', 'text', `Total $${(f?.price ?? 0) + world.details.bags * 35 + (world.details.insurance ? 49 : 0) + (world.flex ? 80 : 0)}`, [60, 224, 400, 44], { tone: 'h1' }))
- out.push(el('confirm', 'button', 'Confirm booking', [60, 330, 240, 56], { tone: 'primary' }))
- if (d > 0.2) out.push(el('flex', 'toggle', 'Upgrade to Flex +$80', [320, 330, 270, 56], { value: world.flex ? 'on' : 'off', tone: 'decoy' }))
- out.push(el('back', 'button', 'Back', [610, 330, 120, 56], { tone: 'quiet' }))
- } else if (world.page === 'promo') {
- out.push(el('h', 'text', 'Hotel deals near you', [60, 84, 600, 44], { tone: 'h1' }))
- out.push(el('p', 'text', 'You left your booking. Nothing here helps with the task.', [60, 140, 800, 34], { tone: 'row' }))
- out.push(el('back-to-booking', 'button', 'Back to my booking', [60, 210, 260, 54], { tone: 'primary' }))
- } else if (world.page === 'done') {
- out.push(el('h', 'text', world.result?.ok ? 'Booked. Exactly as asked.' : 'Booked, but not what was asked.', [60, 110, 880, 50], { tone: world.result?.ok ? 'ok' : 'bad' }))
- out.push(el('p', 'text', world.result?.why ?? '', [60, 176, 880, 34], { tone: 'row' }))
- }
-
- if (world.overlay?.kind === 'cookie') {
- out.push(el('cookie', 'card', '', [0, 486, 1000, 134], { tone: 'overlay' }))
- out.push(el('cookie-text', 'text', 'We value your privacy. This made-up site uses made-up cookies.', [40, 512, 600, 40], { tone: 'overlaytext' }))
- out.push(el('cookie-accept', 'button', 'Accept all', [690, 524, 130, 46], { tone: 'primary', overlay: true }))
- out.push(el('cookie-reject', 'button', 'Reject', [836, 524, 120, 46], { tone: 'quiet', overlay: true }))
- } else if (world.overlay?.kind === 'popup') {
- const [x, y] = world.overlay.at
- out.push(el('popup', 'card', '', [x, y, 480, 300], { tone: 'overlay' }))
- out.push(el('popup-text', 'text', 'Wait! 20% off hotels if you book in the next 10 minutes.', [x + 30, y + 60, 420, 80], { tone: 'overlaytext' }))
- out.push(el('popup-claim', 'button', 'Claim my offer', [x + 40, y + 210, 190, 50], { tone: 'decoy', overlay: true }))
- out.push(el('popup-no', 'button', 'No thanks', [x + 250, y + 210, 170, 50], { tone: 'quiet', overlay: true }))
- out.push(el('popup-close', 'button', '×', [x + 430, y + 12, 36, 36], { tone: 'quiet', overlay: true }))
- }
- // While an overlay is up, the page under it cannot be used.
- if (world.overlay) for (const e of out) if (e.act && !e.overlay) { e.blocked = true }
- return out
-}
-
-const URLS = { search: '/', results: '/flights', details: '/passenger', review: '/review', promo: '/deals/hotels', done: '/confirmed' }
-export const pageUrl = (world, cfg) => `https://${String(cfg.site).toLowerCase().replace(/[^a-z0-9]+/g, '')}.example${URLS[world.page] ?? '/'}`
-
-/** The text Jev reads: the task, the page, and every element it may act on. Nothing hidden. */
-export function observe(world, cfg) {
- const els = elements(world, cfg)
- const can = els.filter((e) => e.act && !e.blocked && !e.disabled)
- const line = (e) => ` ${e.id} ${e.role} "${e.label}"${e.value !== undefined ? ` value "${e.value}"` : ''}${e.context ? ` — ${e.context}` : ''}${e.note ? ` (${e.note})` : ''}`
- const facts = els.filter((e) => e.role === 'text' && !['logo'].includes(e.tone)).map((e) => ` ${e.label}`)
- const text = [
- `You operate a web browser on a travel site. ${cfg.style}`,
- `TASK: ${world.goal}`,
- `PAGE: ${world.page} url ${pageUrl(world, cfg)}`,
- world.overlay ? `OVERLAY: a ${world.overlay.kind === 'cookie' ? 'cookie banner' : 'pop-up'} covers the page. Only its buttons work until it is closed.` : 'OVERLAY: none',
- 'PAGE TEXT:', ...facts,
- 'ELEMENTS you can act on (clicking an input fills it with the right value from the task):', ...can.map(line),
- `RECENT: ${world.history.slice(-4).map((h) => h.id + (h.note ? ` (${h.note})` : '')).join(', ') || 'nothing yet'}`,
- ].join('\n')
- const options = Object.fromEntries(can.map((e) => [e.id, `${e.role} "${e.label}"${e.value !== undefined ? ` value "${e.value}"` : ''}${e.context ? ` — ${e.context}` : ''}`]))
- return { text, options, els }
-}
-
-// ---------------------------------------------------------------- acting
-const inside = (r, x, y) => x >= r[0] && y >= r[1] && x <= r[0] + r[2] && y <= r[1] + r[3]
-
-/**
- * One step. Jev chose `id` from the page it READ. Then the page may shift before the click lands
- * (a pop-up opens, a sponsored row pushes the list down). The click goes to the chosen element's
- * old position and hits whatever is there now. That gap between reading and clicking is the dial.
- */
-export function act(world, cfg, id) {
- if (world.status !== 'running') return null
- world.startedAt ??= world.steps
- const before = elements(world, cfg)
- const target = before.find((e) => e.id === id && e.act)
- world.steps++
- world.shake = null
- const rec = { step: world.steps, id, note: '', x: 500, y: 310, landed: id, shift: null }
- if (!target) { rec.note = 'no such element'; world.wasted++; world.history.push(rec); world.lastAction = rec; return rec }
- rec.x = target.rect[0] + target.rect[2] / 2; rec.y = target.rect[1] + target.rect[3] / 2
-
- // The layout shift, decided before the click lands.
- const d = cfg.distraction
- if (!world.overlay && world.page !== 'done' && world.rng() < d * 0.22) {
- world.overlay = { kind: 'popup', at: [120 + Math.floor(world.rng() * 400), 90 + Math.floor(world.rng() * 200)] }
- rec.shift = 'pop-up'
- } else if (world.page === 'results' && !world.sponsoredShown && world.rng() < d * 0.5) {
- const t = world.task
- world.sponsoredRow = { id: 'ad', airline: 'FlyCheap', dep: 6 * 60 + 10, arr: 6 * 60 + 10 + 300, stops: 0, price: 89, to: NEAR[t.to] ?? 'XXX', sponsored: true }
- world.sponsoredShown = true
- rec.shift = 'sponsored row'
- }
- let hit = target
- if (rec.shift) {
- const now = elements(world, cfg)
- hit = now.filter((e) => e.act && !e.blocked && inside(e.rect, rec.x, rec.y)).pop() ?? null
- if (!hit || hit.id !== id) { world.misclicks++; rec.note = hit ? `layout shifted: landed on ${hit.id}` : 'layout shifted: the click hit nothing' }
- rec.landed = hit?.id ?? null
- }
- if (!hit) { world.wasted++; world.history.push(rec); world.lastAction = rec; return finishIfOver(world, cfg, rec) }
- if (hit.blocked || hit.disabled) { rec.note ||= hit.disabled ? 'disabled' : 'blocked by the overlay'; world.wasted++; world.shake = hit.id; world.history.push(rec); world.lastAction = rec; return finishIfOver(world, cfg, rec) }
- apply(world, hit.id, rec)
- world.history.push(rec); if (world.history.length > 80) world.history.shift()
- world.lastAction = rec
- return finishIfOver(world, cfg, rec)
-}
-
-function apply(world, id, rec) {
- const t = world.task
- const useful = (ok) => { if (!ok) world.wasted++ }
- if (id === 'cookie-accept' || id === 'cookie-reject' || id === 'popup-no' || id === 'popup-close') { world.overlay = null; return }
- if (id === 'popup-claim') { world.overlay = null; world.returnTo = world.page; world.page = 'promo'; rec.note ||= 'left the booking'; world.wasted++; return }
- if (id === 'back-to-booking') { world.page = world.returnTo ?? 'search'; return }
- if (id.startsWith('nav-') || id === 'search-hotels' || id === 'search-cars') { world.returnTo = world.page; world.page = 'promo'; rec.note ||= 'left the booking'; world.wasted++; return }
- if (world.page === 'search') {
- if (id === 'from') { useful(!world.form.from); world.form.from = t.from; rec.typed = t.from }
- else if (id === 'to') { useful(!world.form.to); world.form.to = t.to; rec.typed = t.to }
- else if (id === 'day') { useful(!world.form.day); world.form.day = t.day; rec.typed = t.day }
- else if (id === 'promo') { world.form.promo = 'SAVE10'; rec.typed = 'SAVE10'; world.wasted++ }
- else if (id === 'search-flights') world.page = 'results'
- } else if (world.page === 'results') {
- if (id === 'nonstop-only') world.nonstopOnly = !world.nonstopOnly
- else if (id === 'sort-price') world.sort = 'price'
- else if (id === 'sort-early') world.sort = 'dep'
- else if (id === 'sort-late') world.sort = 'late'
- else if (id.startsWith('select-')) { const fid = id.slice(7); world.chosen = fid === 'ad' ? world.sponsoredRow : world.flights.find((f) => f.id === fid) ?? null; if (world.chosen) world.page = 'details' }
- } else if (world.page === 'details') {
- if (id === 'name') { useful(!world.details.name); world.details.name = t.name; rec.typed = t.name }
- else if (id === 'email') { useful(!world.details.email); world.details.email = t.email; rec.typed = t.email }
- else if (id === 'bag-add') world.details.bags = Math.min(3, world.details.bags + 1)
- else if (id === 'bag-remove') world.details.bags = Math.max(0, world.details.bags - 1)
- else if (id === 'insurance') world.details.insurance = !world.details.insurance
- else if (id === 'newsletter') world.details.newsletter = !world.details.newsletter
- else if (id === 'continue') world.page = 'review'
- else if (id === 'change-flight') { world.page = 'results'; world.chosen = null }
- } else if (world.page === 'review') {
- if (id === 'flex') world.flex = !world.flex
- else if (id === 'back') world.page = 'details'
- else if (id === 'confirm') {
- const f = world.chosen, b = world.best
- const problems = []
- if (!f || !b || f.id !== b.id) problems.push(f?.sponsored ? `booked a sponsored flight to ${f.to}, not ${t.to}` : `booked ${f ? '$' + f.price + ' departing ' + hhmm(f.dep) : 'nothing'}, the ${t.pick}${t.nonstop ? ' nonstop' : ''} was $${b?.price} departing ${b ? hhmm(b.dep) : '?'}`)
- if (world.details.bags !== t.bags) problems.push(`${world.details.bags} bags instead of ${t.bags}`)
- if (world.details.insurance) problems.push('bought insurance')
- if (world.flex) problems.push('bought the Flex upgrade')
- world.result = { ok: problems.length === 0, why: problems.length ? problems.join('; ') : `${t.pick}${t.nonstop ? ' nonstop' : ''} flight, right passenger, ${t.bags} bag(s), no extras` }
- world.page = 'done'; world.status = 'done'; world.finishedAt = world.steps
- }
- }
-}
-
-function finishIfOver(world, cfg, rec) {
- if (world.status === 'running' && world.steps >= cfg.maxSteps) {
- world.status = 'done'; world.finishedAt = world.steps
- world.result = { ok: false, why: `gave up after ${cfg.maxSteps} steps on the ${world.page} page` }
- }
- return rec
-}
diff --git a/store/agents/jev-browser/viewer/studio.css b/store/agents/jev-browser/viewer/studio.css
index 657125459..06e787fa0 100644
--- a/store/agents/jev-browser/viewer/studio.css
+++ b/store/agents/jev-browser/viewer/studio.css
@@ -1,87 +1,90 @@
-/* Jev Browser — blue accent over the shared base. */
:root { --accent: #60a5fa; --accent2: #1d4ed8; }
-.stat.big span { color: var(--accent); }
-#s-ok { color: var(--ok); } #s-mis { color: var(--bad); }
-#stage { gap: 10px; }
-.task { padding: 10px 14px; flex: none; }
-.task-row { display: flex; gap: 10px; align-items: baseline; }
-.goal { font: 600 14px/1.45 var(--sans); color: var(--ink); }
-.goal b { color: var(--accent); font-weight: 700; }
-.task-meters { display: flex; gap: 18px; margin-top: 6px; color: var(--dim); font-size: 12px; align-items: baseline; }
-.task-meters b { color: var(--ink); font-size: 17px; font-variant-numeric: tabular-nums; }
-.task-meters .note { color: var(--bad); font-weight: 600; min-height: 1em; }
+main { display: grid; grid-template-columns: minmax(0, 1fr) 390px; gap: 12px; padding: 12px; height: calc(100vh - 58px); box-sizing: border-box; overflow: hidden; }
+#stage { display: flex; flex-direction: column; gap: 10px; min-width: 0; min-height: 0; }
+#rail { display: flex; flex-direction: column; gap: 10px; overflow: auto; padding-right: 2px; }
-.window { flex: 1 1 auto; min-height: 260px; display: flex; flex-direction: column; border: 1px solid var(--line2); border-radius: 12px; overflow: hidden; background: #0d1017; box-shadow: 0 18px 60px rgba(0,0,0,.5); }
-.chrome { display: flex; align-items: center; gap: 12px; padding: 8px 12px; background: linear-gradient(180deg, #1b2030, #141826); border-bottom: 1px solid var(--line2); flex: none; }
-.lights { display: flex; gap: 6px; } .lights i { width: 11px; height: 11px; border-radius: 50%; background: #ef4444; display: block; } .lights i:nth-child(2) { background: #f59e0b; } .lights i:nth-child(3) { background: #22c55e; }
-.tab { display: flex; align-items: center; gap: 7px; padding: 5px 12px; border-radius: 8px 8px 0 0; background: #0d1017; color: var(--ink); font: 600 12px var(--sans); white-space: nowrap; }
-.fav { width: 12px; height: 12px; border-radius: 3px; background: linear-gradient(135deg, #38bdf8, #6366f1); display: inline-block; }
-.url { flex: 1; min-width: 0; display: flex; align-items: center; gap: 8px; padding: 5px 12px; border-radius: 999px; background: #0a0c12; border: 1px solid var(--line2); color: var(--dim); font-size: 12px; overflow: hidden; white-space: nowrap; text-overflow: ellipsis; }
-.lock { color: var(--ok); font-size: 8px; }
+/* ---- the address bar of the real browser ---- */
+.address { flex: none; display: flex; align-items: center; gap: 8px; padding: 7px 10px; border: 1px solid var(--line); border-radius: 999px; background: var(--panel); }
+.address .lock { color: var(--ok); font-size: 13px; }
+.address input { flex: 1; min-width: 0; background: transparent; border: 0; outline: none; color: var(--ink); font: 12.5px var(--mono); }
+.address input::placeholder { color: var(--faint); }
+.address button { padding: 4px 12px; font-size: 11px; }
-.viewport { position: relative; flex: 1; min-height: 0; overflow: hidden; background: #f4f6fb; }
-.page { position: absolute; left: 0; top: 0; width: 1000px; height: 620px; transform-origin: 0 0; font-family: var(--sans); color: #0f172a; background: linear-gradient(180deg, #e8eefc 0, #f4f6fb 180px); }
-.page::before { content: ''; position: absolute; left: 0; top: 0; width: 1000px; height: 58px; background: #fff; border-bottom: 1px solid #dbe2f0; }
+/* ---- the live browser ---- */
+.screen { position: relative; flex: 1 1 46%; min-height: 200px; border-radius: var(--radius); border: 1px solid var(--line); background: #0a0c12; overflow: hidden; display: grid; place-items: center; }
+.screen img { width: 100%; height: 100%; object-fit: contain; object-position: top center; display: block; }
+.screen img.hidden { display: none; }
+.screen-idle { position: absolute; inset: 0; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 8px; text-align: center; padding: 30px 48px; }
+.screen-idle.hidden { display: none; }
+.screen-idle b { font-size: 19px; color: var(--ink); }
+.screen-idle span { font-size: 12.5px; line-height: 1.6; color: var(--dim); max-width: 620px; }
+.screen-note { margin-top: 6px; font-size: 12px; line-height: 1.5; max-width: 620px; }
+.screen-note.warn { color: var(--warn); }
+.screen-note.bad { color: var(--bad); }
-.el { position: absolute; box-sizing: border-box; display: flex; align-items: center; transition: top .13s ease, left .13s ease, opacity .12s, background .12s, box-shadow .12s; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
-.el.act { cursor: pointer; }
-.el.text { font-size: 15px; color: #334155; }
-.el.t-logo { font: 800 22px var(--sans); color: #1d4ed8; letter-spacing: -.5px; }
-.el.t-h1 { font: 800 28px var(--sans); color: #0f172a; letter-spacing: -.5px; }
-.el.t-ok { font: 800 30px var(--sans); color: #047857; } .el.t-bad { font: 800 30px var(--sans); color: #b91c1c; }
-.el.t-row { background: #fff; border: 1px solid #dbe2f0; border-radius: 10px; padding: 0 14px; font: 500 14.5px var(--sans); color: #1e293b; }
-.el.t-sponsored { background: #fffbeb; border: 1px solid #fcd34d; border-radius: 10px; padding: 0 14px; font: 600 14.5px var(--sans); color: #92400e; }
-.el.t-label { font: 600 16px var(--sans); color: #0f172a; }
-.el.card { background: #fff; border: 1px solid #dbe2f0; border-radius: 16px; box-shadow: 0 10px 30px rgba(15,23,42,.08); }
-.el.card.t-overlay { background: #0f172a; border: none; border-radius: 16px; box-shadow: 0 30px 80px rgba(2,6,23,.55); z-index: 5; }
-.el.t-overlaytext { color: #e2e8f0; font: 600 17px/1.4 var(--sans); white-space: normal; z-index: 6; }
-.el.input { background: #fff; border: 1.5px solid #cbd5e1; border-radius: 10px; padding: 16px 14px 0; font: 600 16px var(--sans); color: #0f172a; align-items: center; }
-.el.input::before { content: attr(data-label); position: absolute; left: 14px; top: 6px; font: 600 10.5px var(--sans); letter-spacing: .4px; text-transform: uppercase; color: #64748b; }
-.el.input.filled { border-color: #22c55e; background: #f0fdf4; }
-.el.button, .el.toggle, .el.link { justify-content: center; border-radius: 10px; font: 700 14.5px var(--sans); border: 1.5px solid #cbd5e1; background: #fff; color: #0f172a; padding: 0 10px; }
-.el.link { border: none; background: none; color: #475569; font-weight: 600; }
-.el.t-primary { background: linear-gradient(135deg, #2563eb, #1d4ed8); border-color: transparent; color: #fff; box-shadow: 0 6px 16px rgba(37,99,235,.35); }
-.el.t-decoy.button, .el.t-decoy.toggle { background: linear-gradient(135deg, #f59e0b, #d97706); border-color: transparent; color: #fff; }
-.el.t-decoy.input { border-style: dashed; }
-.el.t-on { background: #dbeafe; border-color: #2563eb; color: #1d4ed8; }
-.el.toggle.is-on { background: #dcfce7; border-color: #16a34a; color: #166534; }
-.el.toggle.is-on.t-decoy { background: #b91c1c; color: #fff; }
-.el.disabled { opacity: .45; filter: grayscale(.4); }
-.el.blocked { filter: blur(1.2px) brightness(.72); pointer-events: none; }
-.el.overlay-el { z-index: 7; }
-.el.shake { animation: shake .28s; }
-@keyframes shake { 20% { transform: translateX(-5px); } 40% { transform: translateX(5px); } 60% { transform: translateX(-3px); } 80% { transform: translateX(3px); } }
-.el.pop { animation: pop .22s ease-out; }
-@keyframes pop { 0% { transform: scale(.9); opacity: 0; } 100% { transform: none; opacity: 1; } }
+/* ---- the rows ---- */
+.results-head { flex: none; display: flex; align-items: baseline; gap: 10px; font-size: 12px; }
+.results-head b { color: var(--ink); font-size: 13px; }
+.results-head .right { margin-left: auto; font-size: 11px; }
+.table-wrap { flex: 1 1 54%; min-height: 120px; overflow: auto; border: 1px solid var(--line); border-radius: var(--radius); background: var(--panel); }
+table { border-collapse: collapse; width: 100%; font-size: 12.5px; }
+thead th { position: sticky; top: 0; z-index: 2; background: var(--panel2); text-align: left; font: 600 10px var(--mono); letter-spacing: 1.1px; text-transform: uppercase; color: var(--dim); padding: 9px 11px; border-bottom: 1px solid var(--line2); white-space: nowrap; }
+tbody td { padding: 8px 11px; border-bottom: 1px solid var(--line); vertical-align: top; max-width: 320px; }
+tbody tr:hover { background: rgba(255, 255, 255, .025); }
+tbody tr.land td { background: color-mix(in srgb, var(--accent) 18%, transparent); }
+tbody td { transition: background .6s ease; }
+th.n, td.n { width: 38px; color: var(--faint); font-variant-numeric: tabular-nums; }
+td .val { display: block; color: var(--ink); overflow: hidden; text-overflow: ellipsis; }
+td.unsure .val { color: var(--warn); }
+td .missing { color: var(--faint); font-style: italic; font-size: 11.5px; }
+td .conf { display: block; height: 2px; margin-top: 5px; border-radius: 2px; background: linear-gradient(to right, var(--accent) var(--p), var(--line2) var(--p)); }
+td.unsure .conf { background: linear-gradient(to right, var(--warn) var(--p), var(--line2) var(--p)); }
+td .yes { color: var(--ok); font-weight: 600; } td .no { color: var(--faint); }
+td.addr { max-width: 200px; }
+td.addr a { color: var(--dim); text-decoration: none; font-size: 11.5px; display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
+td.addr a:hover { color: var(--accent); text-decoration: underline; }
-/* Jev's mind, drawn on the page: a ring whose strength is the probability of acting on that element. */
-.ring { position: absolute; border-radius: 12px; pointer-events: none; z-index: 8; border: 2.5px solid rgba(167,139,250,.9); box-shadow: 0 0 0 4px rgba(167,139,250,.18), 0 0 22px rgba(167,139,250,.55); transition: opacity .1s; }
-.ring.top { border-color: #22c55e; box-shadow: 0 0 0 4px rgba(34,197,94,.2), 0 0 26px rgba(34,197,94,.7); }
-.ring b { position: absolute; right: 10px; top: -11px; font: 700 11px var(--mono); padding: 1px 6px; border-radius: 6px; background: #4c1d95; color: #fff; }
-.ring.top b { background: #15803d; }
+/* ---- the job ---- */
+.job-body { display: flex; flex-direction: column; gap: 6px; }
+.job-task { font-size: 13px; color: var(--ink); line-height: 1.45; }
+.job-line { font-size: 11.5px; color: var(--dim); overflow: hidden; text-overflow: ellipsis; }
+.job-line b { color: var(--faint); font-weight: 600; }
+.job-line.keep { color: var(--cyan); }
+.job-line .mono { font-family: var(--mono); }
+.job-fields { display: flex; flex-wrap: wrap; gap: 5px; margin-top: 2px; }
+.job-field { display: inline-flex; align-items: baseline; gap: 5px; padding: 3px 8px; border-radius: 999px; border: 1px solid var(--line2); background: #0b0d14; }
+.job-field b { font-size: 11px; font-weight: 600; color: var(--ink); }
+.job-field i { font-style: normal; font-size: 9.5px; color: var(--faint); }
-.cursor { position: absolute; left: 0; top: 0; z-index: 20; pointer-events: none; filter: drop-shadow(0 3px 5px rgba(0,0,0,.45)); will-change: transform; }
-.ripple { position: absolute; width: 16px; height: 16px; margin: -8px 0 0 -8px; border-radius: 50%; border: 3px solid #2563eb; pointer-events: none; z-index: 19; animation: ripple .45s ease-out forwards; }
-.ripple.miss { border-color: #dc2626; }
-@keyframes ripple { 0% { transform: scale(.4); opacity: 1; } 100% { transform: scale(5); opacity: 0; } }
-.shiftnote { position: absolute; z-index: 21; font: 800 13px var(--sans); color: #fff; background: #dc2626; padding: 3px 9px; border-radius: 8px; pointer-events: none; animation: rise .9s ease-out forwards; white-space: nowrap; }
-@keyframes rise { 0% { transform: translateY(6px); opacity: 0; } 15% { opacity: 1; } 100% { transform: translateY(-26px); opacity: 0; } }
+/* ---- Jev on the links ---- */
+.link-list { display: flex; flex-direction: column; gap: 2px; max-height: 260px; overflow: auto; }
+.link-row { position: relative; display: flex; align-items: center; gap: 8px; padding: 4px 8px; border-radius: 6px; background: rgba(255, 255, 255, .02); overflow: hidden; }
+.link-bar { position: absolute; left: 0; top: 0; bottom: 0; background: color-mix(in srgb, var(--dim) 22%, transparent); transition: width .4s ease; }
+.link-row.on .link-bar { background: color-mix(in srgb, var(--accent) 30%, transparent); }
+.link-label { position: relative; flex: 1; min-width: 0; font-size: 11.5px; color: var(--dim); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
+.link-row.on .link-label { color: var(--ink); }
+.link-p { position: relative; flex: none; font-size: 10.5px; color: var(--faint); font-variant-numeric: tabular-nums; }
+.link-row.on .link-p { color: var(--accent); }
-.stamp { position: absolute; inset: 0; z-index: 30; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 10px; text-align: center; backdrop-filter: blur(3px); animation: pop .25s ease-out; padding: 0 24px; }
-.stamp.ok { background: rgba(4,120,87,.80); } .stamp.bad { background: rgba(153,27,27,.82); }
-.stamp h2 { margin: 0; font: 800 clamp(26px, 5vw, 54px) var(--sans); color: #fff; letter-spacing: -1px; }
-.stamp p { margin: 0; font: 600 clamp(13px, 1.6vw, 18px) var(--sans); color: rgba(255,255,255,.92); max-width: 80%; }
-.stamp .nums { font: 700 clamp(14px, 2vw, 22px) var(--mono); color: #fff; }
+/* ---- the feed ---- */
+.feed { display: flex; flex-direction: column; gap: 3px; max-height: 240px; overflow: auto; }
+.feed-row { display: flex; gap: 8px; align-items: baseline; font-size: 11.5px; color: var(--dim); line-height: 1.45; }
+.feed-dot { flex: none; width: 6px; height: 6px; border-radius: 50%; background: var(--line2); margin-top: 5px; }
+.feed-row.row .feed-dot { background: var(--ok); }
+.feed-row.judge .feed-dot { background: var(--accent); }
+.feed-row.bad .feed-dot { background: var(--bad); }
+.feed-row.bad .feed-text { color: var(--bad); }
+.feed-row.done .feed-dot, .feed-row.start .feed-dot { background: var(--cyan); }
+.feed-text { min-width: 0; overflow-wrap: anywhere; }
+.small { font-size: 11.5px; line-height: 1.5; }
+.guard { line-height: 1.55; }
-.toolbar { display: flex; align-items: center; gap: 14px; flex-wrap: wrap; padding: 9px 12px; flex: none; }
-.toolbar .slider { display: flex; align-items: center; gap: 8px; color: var(--dim); font-size: 12px; white-space: nowrap; }
-.toolbar .slider input { width: 120px; } .toolbar .slider b { color: var(--warn); min-width: 30px; font-variant-numeric: tabular-nums; }
-.toolbar .hint { color: var(--dim); font-size: 11.5px; flex: 1 1 220px; } .toolbar .hint b { color: var(--ink); }
+.toast { position: fixed; left: 50%; bottom: 20px; transform: translateX(-50%); z-index: 95; max-width: 70vw; padding: 10px 16px; border-radius: 10px; font-size: 12.5px; color: var(--ink); background: var(--panel2); border: 1px solid color-mix(in srgb, var(--ok) 55%, transparent); box-shadow: 0 10px 34px rgba(0, 0, 0, .5); }
+.toast.bad { border-color: color-mix(in srgb, var(--bad) 60%, transparent); color: #ffd5dc; }
+.toast.hidden { display: none; }
+.cfg-error { margin: 0 12px; padding: 8px 12px; border-radius: 8px; font-size: 12px; color: var(--bad); border: 1px solid color-mix(in srgb, var(--bad) 45%, transparent); background: color-mix(in srgb, var(--bad) 8%, transparent); }
+.cfg-error.hidden { display: none; }
+.hidden { display: none; }
-.statetext { margin: 0; font: 10.5px/1.5 var(--mono); color: #cbd5e1; white-space: pre-wrap; word-break: break-word; max-height: 250px; overflow: auto; }
-.statetext .k { color: var(--accent); font-weight: 700; } .statetext .pick { color: #0b0d14; background: var(--ok); border-radius: 3px; } .statetext .warn { color: var(--bad); }
-#feed { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 5px; max-height: 210px; overflow-y: auto; font-size: 11.5px; color: var(--dim); }
-#feed li { display: grid; grid-template-columns: 16px 1fr; gap: 6px; line-height: 1.4; }
-#feed li.ok i { color: var(--ok); } #feed li.bad i { color: var(--bad); } #feed li i { font-style: normal; font-weight: 800; } #feed li b { color: var(--ink); font-weight: 600; }
-.task-row .pill { white-space: nowrap; flex: none; }
+@media (max-width: 1100px) { main { grid-template-columns: 1fr; height: auto; overflow: auto; } #rail { max-height: none; } }
diff --git a/store/agents/jev-browser/viewer/studio.js b/store/agents/jev-browser/viewer/studio.js
index ec2bef680..4df0428b3 100644
--- a/store/agents/jev-browser/viewer/studio.js
+++ b/store/agents/jev-browser/viewer/studio.js
@@ -1,175 +1,197 @@
-// Jev Browser pane. The server sends one frame per step: the action Jev just chose (with the full
-// probability over the elements it could have chosen) and the page as it is now. This file plays
-// that as: rings on the old page showing Jev's mind -> the cursor flies to the chosen spot -> a click
-// ripple (red when a layout shift made it land somewhere else) -> the new page.
-'use strict'
-
+// studio.js — the Jev Browser pane: the live browser, what Jev decided about every link on the
+// page, and the rows as they land. Everything comes from the viewer over SSE; the pane asks for
+// nothing the desktop web view cannot give it.
const $ = (id) => document.getElementById(id)
-const page = $('page'), viewport = $('viewport'), cursor = $('cursor'), stamp = $('stamp')
-const PAGE_W = 1000, PAGE_H = 620
-let scale = 1, showMind = true, paused = false
-let cur = { x: 500, y: 320 }, anim = null, pending = null
-let lastTaskKey = '', costAtTaskStart = 0, costNow = 0, lastFeedLen = -1
-const nodes = new Map() // element id -> DOM node
+const h = (tag, cls, text) => { const n = document.createElement(tag); if (cls) n.className = cls; if (text != null) n.textContent = text; return n }
+const fmtN = (n) => Number(n || 0).toLocaleString('en-US')
+const fmtCost = (v) => (v <= 0 ? '$0' : v < 0.01 ? `$${v.toFixed(5)}` : `$${v.toFixed(4)}`)
+const pct = (p) => `${Math.round(p * 100)}%`
+
+let S = null, shownRows = 0, toastTimer = null
+const tw = { rows: { v: 0, t: 0 }, pages: { v: 0, t: 0 }, links: { v: 0, t: 0 }, rate: { v: 0, t: 0 }, cost: { v: 0, t: 0 } }
-function fit() {
- const r = viewport.getBoundingClientRect()
- scale = Math.min(r.width / PAGE_W, r.height / PAGE_H)
- page.style.transform = `scale(${scale})`
- page.style.left = Math.max(0, (r.width - PAGE_W * scale) / 2) + 'px'
- placeCursor()
+async function post(cmd, body = {}) {
+ try {
+ const r = await fetch('/control', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ cmd, ...body }) })
+ return await r.json()
+ } catch { return { ok: false, error: 'the viewer is restarting' } }
+}
+function toast(msg, bad) {
+ const t = $('toast')
+ t.textContent = msg; t.className = 'toast' + (bad ? ' bad' : '')
+ clearTimeout(toastTimer); toastTimer = setTimeout(() => t.classList.add('hidden'), bad ? 7000 : 4000)
}
-new ResizeObserver(fit).observe(viewport)
-const placeCursor = () => { const off = parseFloat(page.style.left) || 0; cursor.style.transform = `translate(${off + cur.x * scale - 3}px, ${cur.y * scale - 2}px)` }
-// ---------------------------------------------------------------- the page
-function renderPage(f) {
- const seen = new Set()
- for (const e of f.els) {
- seen.add(e.id)
- let n = nodes.get(e.id)
- if (!n) {
- n = document.createElement('div'); n.dataset.id = e.id; nodes.set(e.id, n); page.appendChild(n)
- n.classList.add('pop')
- n.addEventListener('click', () => { if (n.classList.contains('act')) post({ cmd: 'click', id: e.id }) })
- }
- const isOn = e.value === 'on'
- n.className = ['el', e.role, e.act ? 'act' : '', e.tone ? 't-' + e.tone : '', e.disabled ? 'disabled' : '', e.blocked ? 'blocked' : '', e.overlay ? 'overlay-el' : '', isOn ? 'is-on' : '', e.role === 'input' && e.value ? 'filled' : '', f.shake === e.id ? 'shake' : '', n.classList.contains('pop') ? 'pop' : ''].filter(Boolean).join(' ')
- const [x, y, w, h] = e.rect
- n.style.left = x + 'px'; n.style.top = y + 'px'; n.style.width = w + 'px'; n.style.height = h + 'px'
- if (e.role === 'input') { n.dataset.label = e.label; if (n.dataset.typing !== '1') n.textContent = e.value || '' }
- else if (e.role === 'toggle') n.textContent = (isOn ? '☑ ' : '☐ ') + e.label
- else n.textContent = e.label
- n.title = e.act ? `${e.id} — click it yourself` : ''
+// ---- the job -------------------------------------------------------------------------------------
+function renderJob() {
+ const box = $('jobBody')
+ box.textContent = ''
+ box.append(h('div', 'job-task', S.task))
+ const start = h('div', 'job-line')
+ start.append(h('b', '', 'starts at '), h('span', 'mono', S.demoUrl && S.start === S.demoUrl ? 'a made-up job board on this machine' : S.start || 'nowhere yet'))
+ box.append(start)
+ const item = h('div', 'job-line')
+ item.append(h('b', '', 'collects '), h('span', '', S.item))
+ box.append(item)
+ const fields = h('div', 'job-fields')
+ for (const f of S.fields) {
+ const chip = h('span', 'job-field')
+ chip.append(h('b', '', f.name), h('i', '', f.type === 'yesno' ? 'yes / no' : f.type === 'score' ? 'scale' : 'off the page'))
+ chip.title = f.ask
+ fields.append(chip)
}
- for (const [id, n] of nodes) if (!seen.has(id)) { n.remove(); nodes.delete(id) }
- setTimeout(() => { for (const n of nodes.values()) n.classList.remove('pop') }, 240)
+ box.append(fields)
+ if (S.keep) box.append(h('div', 'job-line keep', `keeps only: ${S.keep}`))
+ box.append(h('div', 'job-line dim', `at most ${fmtN(S.maxItems)} things from ${fmtN(S.maxPages)} pages`))
}
-function clearRings() { for (const r of page.querySelectorAll('.ring')) r.remove() }
-function drawRings(f) {
- clearRings()
- if (!showMind || !f.last?.probs) return
- const top = Object.entries(f.last.probs).sort((a, b) => b[1] - a[1]).slice(0, 4)
- top.forEach(([id, p], i) => {
- const n = nodes.get(id); if (!n || p < 0.03) return
- const r = document.createElement('div'); r.className = 'ring' + (i === 0 ? ' top' : '')
- r.style.left = parseFloat(n.style.left) - 4 + 'px'; r.style.top = parseFloat(n.style.top) - 4 + 'px'
- r.style.width = parseFloat(n.style.width) + 8 + 'px'; r.style.height = parseFloat(n.style.height) + 8 + 'px'
- r.style.opacity = String(0.35 + 0.65 * Math.min(1, p * 1.4))
- const b = document.createElement('b'); b.textContent = p >= 0.995 ? '1.00' : p.toFixed(2).replace(/^0/, ''); r.appendChild(b)
- page.appendChild(r)
- })
+// ---- the results table -----------------------------------------------------------------------------
+function renderHead() {
+ const tr = $('resultsHead')
+ tr.textContent = ''
+ tr.append(h('th', 'n', '#'))
+ for (const f of S.fields) tr.append(h('th', '', f.name))
+ if (S.keep) tr.append(h('th', '', 'matches'))
+ tr.append(h('th', 'addr', 'page'))
}
-
-function ripple(x, y, miss, note) {
- const r = document.createElement('div'); r.className = 'ripple' + (miss ? ' miss' : ''); r.style.left = x + 'px'; r.style.top = y + 'px'
- page.appendChild(r); setTimeout(() => r.remove(), 480)
- if (note) { const s = document.createElement('div'); s.className = 'shiftnote'; s.textContent = note; s.style.left = Math.min(PAGE_W - 220, x + 14) + 'px'; s.style.top = Math.max(8, y - 30) + 'px'; page.appendChild(s); setTimeout(() => s.remove(), 920) }
+function cellFor(row, f) {
+ const td = h('td')
+ const v = row.fields[f.id] ?? ''
+ const c = row.confidence[f.id] ?? 0
+ if (!v) { td.append(h('span', 'missing', 'not on the page')); return td }
+ td.append(h('span', 'val', v))
+ const bar = h('i', 'conf')
+ bar.style.setProperty('--p', `${Math.round(c * 100)}%`)
+ bar.title = `Jev's confidence: ${pct(c)}`
+ if (c < 0.65) td.classList.add('unsure')
+ td.append(bar)
+ return td
}
-
-function typeInto(id, text, ms) {
- const n = nodes.get(id); if (!n || !text) return
- n.dataset.typing = '1'; n.textContent = ''
- const t0 = performance.now()
- const tick = () => { const k = Math.min(text.length, Math.ceil(((performance.now() - t0) / ms) * text.length)); n.textContent = text.slice(0, k); if (k < text.length) requestAnimationFrame(tick); else n.dataset.typing = '0' }
- tick()
+function renderRows() {
+ const body = $('resultsBody')
+ if (shownRows > S.rows.length) { body.textContent = ''; shownRows = 0 }
+ for (let i = shownRows; i < S.rows.length; i++) {
+ const row = S.rows[i]
+ const tr = h('tr', 'land')
+ tr.append(h('td', 'n', String(row.n)))
+ for (const f of S.fields) tr.append(cellFor(row, f))
+ if (S.keep) { const td = h('td'); td.append(h('span', row.keep ? 'yes' : 'no', row.keep ? 'yes' : 'no')); tr.append(td) }
+ const a = h('a', '', row.title || row.url)
+ a.href = row.url; a.target = '_blank'; a.rel = 'noreferrer'; a.title = row.url
+ const td = h('td', 'addr'); td.append(a); tr.append(td)
+ body.append(tr)
+ setTimeout(() => tr.classList.remove('land'), 40)
+ }
+ shownRows = S.rows.length
+ $('resultsNote').textContent = S.rows.length ? `${fmtN(S.rows.length)} so far · saved to ${S.resultsFile} as they land` : 'nothing yet'
}
-// ---------------------------------------------------------------- one frame = one played step
-function play(f) {
- const a = f.lastAction
- const isNewStep = a && (!play.lastStep || a.step !== play.lastStep || f.task.goal !== play.lastGoal)
- play.lastStep = a?.step; play.lastGoal = f.task.goal
- if (!isNewStep) { if (!anim) { clearRings(); renderPage(f) } return }
- // 1. rings on the page Jev read, 2. fly, 3. click, 4. show the page as it is now
- drawRings(f)
- const dwell = paused ? 900 : 0 // stepping by hand: time to read Jev's mind on the page it read
- const from = { ...cur }, to = { x: a.x, y: a.y }, dur = Math.max(50, Math.min(150, f.stepMs * 0.55)), t0 = performance.now() + dwell
- if (anim) cancelAnimationFrame(anim)
- const stepAnim = (now) => {
- if (now < t0) { anim = requestAnimationFrame(stepAnim); return }
- const k = Math.min(1, (now - t0) / dur), e = 1 - Math.pow(1 - k, 3)
- cur = { x: from.x + (to.x - from.x) * e, y: from.y + (to.y - from.y) * e - Math.sin(k * Math.PI) * Math.min(40, Math.hypot(to.x - from.x, to.y - from.y) * 0.12) }
- placeCursor()
- if (k < 1) { anim = requestAnimationFrame(stepAnim); return }
- anim = null
- const miss = !!a.shift && a.landed !== a.id
- ripple(a.x, a.y, miss || /disabled|blocked|no such/.test(a.note || ''), a.shift ? (miss ? `layout shifted: ${a.shift}` : a.shift) : '')
- clearRings()
- renderPage(f)
- if (a.typed) typeInto(a.landed, a.typed, Math.max(40, f.stepMs * 0.35))
+// ---- Jev on the links --------------------------------------------------------------------------------
+function renderLinks() {
+ const box = $('linkList')
+ box.textContent = ''
+ if (!S.links.length) { box.append(h('div', 'dim small', 'Nothing judged yet.')); return }
+ const yes = S.links.filter((l) => l.p >= 0.5).length
+ $('linkNote').textContent = `${fmtN(S.links.length)} shown · ${fmtN(yes)} are items`
+ for (const l of S.links) {
+ const row = h('div', 'link-row' + (l.p >= 0.5 ? ' on' : ''))
+ const bar = h('i', 'link-bar')
+ bar.style.width = `${Math.max(2, Math.round(l.p * 100))}%`
+ row.append(bar, h('span', 'link-label', l.label), h('span', 'link-p', pct(l.p)))
+ row.title = l.label
+ box.append(row)
}
- anim = requestAnimationFrame(stepAnim)
}
-// ---------------------------------------------------------------- everything around the page
-const fmtMoney = (v) => v <= 0 ? '$0' : v < 0.0001 ? '<$.0001' : v < 1 ? '$' + v.toFixed(4).replace(/^0/, '') : '$' + v.toFixed(2)
-const esc = (s) => String(s).replace(/[&<>]/g, (c) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;' }[c]))
-
-function paintGoal(f) {
- const g = esc(f.task.goal).replace(/\b(cheapest|earliest|latest|nonstop)\b/g, '<b>$1</b>').replace(/from (\S+) to (\S+) on (\S+)/, 'from <b>$1</b> to <b>$2</b> on <b>$3</b>')
- $('goal').innerHTML = g
- $('taskNo').textContent = `TASK ${f.task.index + 1}/${f.task.count}`
+// ---- the feed ----------------------------------------------------------------------------------------
+function renderFeed() {
+ const box = $('feed')
+ box.textContent = ''
+ for (const e of S.feed.slice(0, 18)) {
+ const row = h('div', `feed-row ${e.kind}`)
+ row.append(h('span', 'feed-dot'), h('span', 'feed-text', e.text))
+ box.append(row)
+ }
+ if (!S.feed.length) box.append(h('div', 'dim small', 'Press Start.'))
}
-function paintStateText(f) {
- const pick = f.last?.choice
- $('stateText').innerHTML = (f.stateText || '').split('\n').slice(1).map((l) => {
- const e = esc(l)
- if (/^(TASK|PAGE|OVERLAY|PAGE TEXT|ELEMENTS|RECENT)/.test(l)) return e.replace(/^([A-Z ]+)/, '<span class="k">$1</span>').replace(/(a (cookie banner|pop-up) covers.*)/, '<span class="warn">$1</span>')
- if (pick && l.startsWith(` ${pick} `)) return `<span class="pick">${e}</span>`
- return e
- }).join('\n')
+
+// ---- the top bar and the state -------------------------------------------------------------------------
+function renderTop() {
+ const p = S.progress
+ tw.rows.t = p.rows; tw.pages.t = p.pages; tw.links.t = p.links; tw.rate.t = p.perSec; tw.cost.t = p.costUsd
+ $('phasePill').textContent = S.phase === 'running' ? 'browsing' : S.phase
+ $('phasePill').className = 'pill ' + (S.phase === 'running' ? 'ok' : S.phase === 'done' ? 'ok' : S.phase === 'stopped' ? 'warn' : '')
+ $('startBtn').classList.toggle('hidden', S.phase === 'running')
+ $('stopBtn').classList.toggle('hidden', S.phase !== 'running')
+ $('rateStat').classList.toggle('hot', S.phase === 'running')
}
-function paintFeed(f) {
- const rs = f.session.results
- if (f.session.tasks === lastFeedLen) return
- lastFeedLen = f.session.tasks
- $('feed').innerHTML = rs.slice().reverse().map((r) => `<li class="${r.ok ? 'ok' : 'bad'}"><i>${r.ok ? '✓' : '✗'}</i><span><b>${(r.wallMs / 1000).toFixed(1)} s · ${r.steps} steps</b> · ${esc(r.ok ? r.goal.replace(/^Book the /, '').replace(/ for .*/, '') : r.why)}</span></li>`).join('') || '<li><i></i><span>The first booking is under way.</span></li>'
+function applyState(s) {
+ const first = !S
+ const sameFields = S && S.fields.length === s.fields.length && S.fields.every((f, i) => f.id === s.fields[i].id && f.name === s.fields[i].name)
+ S = s
+ $('taskLine').textContent = s.task
+ $('cfgError').classList.toggle('hidden', !(s.error || s.jevError))
+ $('cfgError').textContent = s.error || (s.jevError ? `Jev: ${s.jevError}` : '')
+ if (!sameFields) { renderHead(); shownRows = 0; $('resultsBody').textContent = '' }
+ renderJob(); renderRows(); renderLinks(); renderFeed(); renderTop()
+ const idle = !s.here.url && !s.rows.length
+ $('screenIdle').classList.toggle('hidden', !idle)
+ $('chromeNote').textContent = !s.chrome.found
+ ? 'Google Chrome was not found on this machine. Install it, or set CHROME_PATH to where it is.'
+ : s.client === 'mock' ? 'No Jev key yet, so an offline stand-in will answer. It only matches words: good enough to watch, not good enough to act on. Paste a key in the panel on the right.' : ''
+ $('chromeNote').className = 'screen-note' + (!s.chrome.found ? ' bad' : s.client === 'mock' ? ' warn' : '')
+ if (document.activeElement !== $('urlInput')) $('urlInput').value = s.here.url || ''
+ $('urlInput').placeholder = s.chrome.open ? 'where the browser is' : 'the browser is not open yet'
+ if (first) $('screen').classList.toggle('hidden', true)
}
-function onFrame(f) {
- paused = !f.running
- const key = `${f.task.round}:${f.task.index}:${f.task.goal}`
- if (key !== lastTaskKey) { lastTaskKey = key; costAtTaskStart = costNow; stamp.classList.add('hidden'); play.lastStep = null }
- play(f)
- paintGoal(f); paintStateText(f); paintFeed(f)
- $('title').textContent = f.title
- $('tabTitle').textContent = `${f.site} · ${f.page}`
- $('url').textContent = f.url
- $('m-time').textContent = (f.wallMs / 1000).toFixed(1); $('m-steps').textContent = f.steps; $('m-cost').textContent = fmtMoney(Math.max(0, costNow - costAtTaskStart))
- $('m-note').textContent = f.lastAction?.note && !/you clicked$/.test(f.lastAction.note) ? f.lastAction.note : ''
- const s = f.session
- $('s-tasks').textContent = s.tasks; $('s-ok').textContent = s.tasks ? Math.round((s.ok / s.tasks) * 100) + '%' : '—'
- $('s-steps').textContent = s.tasks ? (s.steps / s.tasks).toFixed(1) : '—'; $('s-secs').textContent = s.tasks ? (s.wallMs / s.tasks / 1000).toFixed(1) : '—'
- $('s-mis').textContent = s.misclicks + f.misclicks
- $('pause').textContent = f.running ? 'Pause' : 'Resume'
- if (document.activeElement !== $('distraction')) { $('distraction').value = f.distraction; $('distractionVal').textContent = Number(f.distraction).toFixed(2) }
- if (document.activeElement !== $('stepMs')) { $('stepMs').value = f.stepMs; $('stepMsVal').textContent = f.stepMs }
- const problem = f.cfgError || f.error
- $('cfgError').classList.toggle('hidden', !problem); $('cfgError').textContent = problem ? `site.json: ${problem} — still running on the last good settings.` : ''
- if (f.status === 'done' && f.result) {
- stamp.className = 'stamp ' + (f.result.ok ? 'ok' : 'bad')
- stamp.innerHTML = `<h2>${f.result.ok ? 'Booked. Exactly right.' : 'Wrong booking'}</h2><div class="nums">${(f.wallMs / 1000).toFixed(1)} s · ${f.steps} steps · ${fmtMoney(Math.max(0, costNow - costAtTaskStart))}</div><p>${esc(f.result.why)}</p>`
- } else stamp.classList.add('hidden')
+// ---- tweened numbers and the frame loop -------------------------------------------------------------------
+function step() {
+ for (const [id, t] of Object.entries(tw)) {
+ if (Math.abs(t.t - t.v) < 0.001) { t.v = t.t } else t.v += (t.t - t.v) * 0.18
+ }
+ $('sRows').textContent = fmtN(Math.round(tw.rows.v))
+ $('sPages').textContent = fmtN(Math.round(tw.pages.v))
+ $('sLinks').textContent = fmtN(Math.round(tw.links.v))
+ $('sRate').textContent = tw.rate.v >= 10 ? Math.round(tw.rate.v) : tw.rate.v.toFixed(1)
+ $('sCost').textContent = fmtCost(tw.cost.v)
+ requestAnimationFrame(step)
}
-const post = (body) => fetch('/control', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) }).catch(() => {})
-$('pause').onclick = () => post({ cmd: paused ? 'start' : 'pause' })
-$('step').onclick = () => post({ cmd: 'tick' })
-$('reset').onclick = () => post({ cmd: 'reset' })
-$('popup').onclick = () => post({ cmd: 'popup' })
-$('nextTask').onclick = () => { const m = /(\d+)\/(\d+)/.exec($('taskNo').textContent); post({ cmd: 'task', index: m ? Number(m[1]) % Number(m[2]) : 0 }) }
-$('heat').onclick = () => { showMind = !showMind; $('heat').classList.toggle('on', showMind); if (!showMind) clearRings() }
-$('distraction').oninput = (e) => { $('distractionVal').textContent = Number(e.target.value).toFixed(2); post({ cmd: 'set', key: 'distraction', value: Number(e.target.value) }) }
-$('stepMs').oninput = (e) => { $('stepMsVal').textContent = e.target.value; post({ cmd: 'set', key: 'stepMs', value: Number(e.target.value) }) }
-
-async function pollJev() {
- try { const s = await (await fetch('/jev', { cache: 'no-store' })).json(); costNow = s.costUsd; $('s-rate').textContent = s.callsPerSec.toFixed(1); $('s-cost').textContent = fmtMoney(s.costUsd) } catch { /* restarting */ }
- setTimeout(pollJev, 400)
+// ---- controls ---------------------------------------------------------------------------------------------
+$('startBtn').addEventListener('click', async () => {
+ $('startBtn').disabled = true
+ const r = await post('start')
+ $('startBtn').disabled = false
+ if (r.ok === false) toast(r.error || 'it could not start', true)
+})
+$('stopBtn').addEventListener('click', () => post('stop'))
+$('resetBtn').addEventListener('click', () => post('reset'))
+$('downloadBtn').addEventListener('click', async () => {
+ await post('export')
+ const a = document.createElement('a'); a.href = '/download/results.csv'; a.download = 'results.csv'
+ document.body.append(a); a.click(); a.remove()
+ toast('results.csv is also in this project folder.')
+})
+async function go() {
+ const url = $('urlInput').value.trim()
+ if (!url) return
+ const r = await post('openHere', { url })
+ if (r.ok === false) toast(r.error, true)
}
+$('goBtn').addEventListener('click', go)
+$('urlInput').addEventListener('keydown', (e) => { if (e.key === 'Enter') { e.preventDefault(); go() } })
-fit()
+// ---- the stream ----------------------------------------------------------------------------------------------
const es = new EventSource('/events')
-es.addEventListener('state', (e) => onFrame(JSON.parse(e.data)))
-pollJev()
+es.addEventListener('state', (e) => applyState(JSON.parse(e.data)))
+es.addEventListener('view', (e) => applyState(JSON.parse(e.data)))
+es.addEventListener('shot', (e) => {
+ const s = JSON.parse(e.data)
+ const img = $('screen')
+ img.src = `data:image/jpeg;base64,${s.jpegBase64}`
+ img.classList.remove('hidden')
+ $('screenIdle').classList.add('hidden')
+})
+window.addEventListener('jev-connected', () => toast('Connected. The next page is read by the real model.'))
+requestAnimationFrame(step)
diff --git a/store/agents/jev-browser/viewer/viewer.mjs b/store/agents/jev-browser/viewer/viewer.mjs
index 7de4687d5..98c486780 100644
--- a/store/agents/jev-browser/viewer/viewer.mjs
+++ b/store/agents/jev-browser/viewer/viewer.mjs
@@ -1,198 +1,276 @@
-// Jev Browser viewer — a loopback server where Jev operates a web browser, live.
-// The site is a small made-up travel shop. At every step the page is written out as text (the task,
-// the page, every element that can be acted on) and Jev (TypeSafe's System One model) picks ONE
-// element out of the list, a new action space every step, plus two yes/no reads, all in one call.
-// A booking takes a few seconds. The honest dial is `distraction`: cookie walls, pop-ups, decoys and
-// layout shifts that land between reading the page and clicking it.
+// viewer.mjs — the Jev Browser viewer server.
//
-// The chat agent edits site.json (the tasks, the distraction, the pace). The pane lets a person
-// click the page themselves, throw a pop-up at Jev, and move the dials.
-//
-// Harness env: HARNESS_VIEWER_PORT, HARNESS_WORKSPACE. Workspace holds site.json (watched live).
-
-import { join, resolve, dirname } from 'node:path'
+// It owns a real Chrome, reads `browse.json`, and walks real web pages: one Jev call per page,
+// which judges every link on it and pulls every field at once. Rows land in `results.csv`.
+// `browse.json` is also the recipe: run it again tomorrow and you get today's answer again.
+import { join, dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
-import { evaluate, jev } from '../toolchain/jev.mjs'
-import { serveViewer, watchConfig, writeVerdict, clean } from './kit.mjs'
-import { DEFAULT, createWorld, observe, act, elements, pageUrl, cleanTask } from './sim.mjs'
+import { writeFileSync, renameSync, mkdirSync } from 'node:fs'
+import { evaluate, PRICE_PER_MTOK, resolveCredentials, describeCredentials } from '../toolchain/jev.mjs'
+import { serveViewer, writeVerdict, watchConfig, clean } from './kit.mjs'
+import { openChrome, findChrome, Refused } from '../toolchain/chrome.mjs'
+import { normalizeJob, runJob, LIMITS } from './crawl.mjs'
import { browserMock } from './mock.mjs'
+import { startDemoSite } from './demosite.mjs'
const HERE = dirname(fileURLToPath(import.meta.url))
-const clampN = (v, lo, hi, d) => { const n = Number(v); return Number.isFinite(n) ? Math.max(lo, Math.min(hi, n)) : d }
+const MARKER = 'browse.json'
+const RESULTS = 'results.csv'
+const FEED = 40
-/** Keep a wild config from breaking the demo. check.mjs reports the same ranges to the agent. */
-export function sanitize(raw) {
- return {
- ...DEFAULT, ...raw,
- title: String(raw.title ?? DEFAULT.title).slice(0, 80),
- description: String(raw.description ?? DEFAULT.description).slice(0, 300),
- site: String(raw.site ?? DEFAULT.site).replace(/[^\w .-]/g, '').slice(0, 24) || DEFAULT.site,
- style: String(raw.style ?? DEFAULT.style).slice(0, 600),
- stepMs: clampN(raw.stepMs, 40, 2000, DEFAULT.stepMs),
- distraction: clampN(raw.distraction, 0, 1, DEFAULT.distraction),
- flights: Math.round(clampN(raw.flights, 4, 9, DEFAULT.flights)),
- maxSteps: Math.round(clampN(raw.maxSteps, 12, 200, DEFAULT.maxSteps)),
- seed: Math.round(clampN(raw.seed, 0, 1e9, DEFAULT.seed)),
- tasks: (Array.isArray(raw.tasks) && raw.tasks.length ? raw.tasks : DEFAULT.tasks).slice(0, 40).map(cleanTask),
- }
-}
+export async function startBrowserViewer({ workspace, port = 0, show = null } = {}) {
+ let server = null, watcher = null, chrome = null, demo = null, running = null, stopFlag = false
+ let job = normalizeJob({}).job, configError = null, jevError = null
+ let rows = [], feed = [], links = [], here = { url: '', title: '' }
+ let counters = { pages: 0, calls: 0, questions: 0, tokens: 0, costUsd: 0, links: 0, errors: 0 }
+ let phase = 'idle', startedAt = 0, finishedAt = 0, lastRun = null
+ let shotTimer = null, shotBusy = false
+ let expecting = { expecting: 'list', item: '' }
+ const mock = browserMock(() => expecting)
+ const liveRoute = () => resolveCredentials()?.provider ?? null
+ const chromePath = findChrome()
-export async function startBrowserViewer({ workspace, port = 0 } = {}) {
- workspace = resolve(workspace)
-
- // Every binding is declared before anything can call into the closures below.
- let world = null
- let stopped = false, running = false, busy = false
- let timer = null, salt = 1, taskIndex = 0, round = 0, lastVerdictAt = 0, taskWallStart = 0, doneAt = 0
- let overrides = {}
- let last = { probs: {}, choice: null, confidence: 0, pDone: 0, pObstacle: 0 }
- let stateText = ''
- let error = null
- let server = null
- const session = { tasks: 0, ok: 0, steps: 0, wasted: 0, misclicks: 0, wallMs: 0, results: [] }
-
- const cfgWatch = watchConfig(join(workspace, 'site.json'), DEFAULT, () => { overrides = {}; taskIndex = 0; round = 0; newTask(); push(true) })
- const cfg = () => sanitize({ ...cfgWatch.get(), ...overrides })
-
- function newTask() {
- world = createWorld(cfg(), taskIndex, round)
- taskWallStart = 0; doneAt = 0
- stateText = observe(world, cfg()).text
- last = { probs: {}, choice: null, confidence: 0, pDone: 0, pObstacle: 0 }
- }
+ // ---- what the pane sees ---------------------------------------------------------------------
+ const view = () => ({
+ task: job.task, item: job.item, start: job.start, fields: job.fields.map((f) => ({ id: f.id, name: f.name, ask: f.ask, type: f.type })),
+ keep: job.keep, maxItems: job.maxItems, maxPages: job.maxPages, limits: LIMITS,
+ error: configError, jevError, client: liveRoute() ?? 'mock', jevSays: describeCredentials(),
+ chrome: { found: !!chromePath, open: !!chrome?.alive },
+ demoUrl: demo?.url ?? null, phase, here,
+ progress: {
+ ...counters,
+ rows: rows.length,
+ elapsedMs: startedAt ? (finishedAt || Date.now()) - startedAt : 0,
+ perSec: counters.questions / Math.max(0.5, ((finishedAt || Date.now()) - startedAt) / 1000 || 1),
+ },
+ links, feed, rows, resultsFile: RESULTS,
+ })
+ const push = (event = 'state') => server?.broadcast(view(), event)
+ const say = (text, kind = 'info') => { feed = [{ at: Date.now(), kind, text: clean(text).slice(0, 220) }, ...feed].slice(0, FEED) }
- function frame() {
- const c = cfg()
- return {
- t: world.steps, stepMs: c.stepMs, title: c.title, site: c.site, description: c.description,
- task: { goal: world.goal, index: world.taskIndex % world.taskCount, count: world.taskCount, round },
- page: world.page, url: pageUrl(world, c), els: elements(world, c), overlay: world.overlay?.kind ?? null, shake: world.shake,
- lastAction: world.lastAction, last,
- status: world.status, result: world.result, steps: world.steps, wasted: world.wasted, misclicks: world.misclicks,
- wallMs: taskWallStart ? (doneAt || Date.now()) - taskWallStart : 0,
- session: { ...session, results: session.results.slice(-12) },
- distraction: c.distraction, maxSteps: c.maxSteps, stateText,
- running, error, cfgError: cfgWatch.error(), overrides,
+ // ---- results.csv ------------------------------------------------------------------------------
+ // A value that starts with = + - @ would run as a formula in a spreadsheet: quote it.
+ const cell = (v) => { let s = v == null ? '' : String(v); if (/^[=+\-@\t\r]/.test(s) && !/^[+-]?\d+(\.\d+)?$/.test(s)) s = "'" + s; return /[",\n\r]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s }
+ function resultsCsv() {
+ const head = ['item', 'page title', 'address', ...job.fields.flatMap((f) => [f.name, `${f.name} confidence`])]
+ if (job.keep) head.push('matches what you asked', 'matches confidence')
+ const lines = [head.map(cell).join(',')]
+ for (const r of rows) {
+ const line = [r.n, r.title, r.url, ...job.fields.flatMap((f) => [r.fields[f.id] ?? '', (r.confidence[f.id] ?? 0).toFixed(2)])]
+ if (job.keep) line.push(r.keep ? 'yes' : 'no', (r.keepConfidence ?? 0).toFixed(2))
+ lines.push(line.map(cell).join(','))
}
+ return lines.join('\n') + '\n'
+ }
+ function saveResults() {
+ try { const f = join(workspace, RESULTS); writeFileSync(f + '.tmp', resultsCsv()); renameSync(f + '.tmp', f) } catch { /* workspace gone */ }
}
+ // ---- the verdict -------------------------------------------------------------------------------
function verdict() {
- const c = cfg()
- const problem = cfgWatch.error() || error
- const rate = session.tasks ? Math.round((session.ok / session.tasks) * 100) : 0
- const avgSteps = session.tasks ? (session.steps / session.tasks).toFixed(1) : '—'
- writeVerdict(workspace, {
- ready: session.tasks > 0 || world.steps > 0,
- summary: problem
- ? `Jev Browser needs a fix: ${problem}`
- : `${c.title} · ${session.tasks} bookings · ${rate}% exactly right · ${avgSteps} steps each · ${session.misclicks} clicks lost to layout shifts`,
- findings: [
- ...(problem ? [{ severity: 'error', kind: 'site', message: problem }] : []),
- ...session.results.filter((r) => !r.ok).slice(-5).map((r) => ({ severity: 'warning', kind: 'booking', message: `${r.goal} → ${r.why}` })),
- { severity: 'info', kind: 'run', message: `distraction ${c.distraction.toFixed(2)}, ${c.stepMs} ms per step, ${session.wasted} wasted steps in ${session.steps}` },
- ],
- artifact: 'site.json',
+ const filled = job.fields.map((f) => {
+ const got = rows.filter((r) => String(r.fields[f.id] ?? '') !== '')
+ const conf = got.length ? got.reduce((s, r) => s + (r.confidence[f.id] ?? 0), 0) / got.length : 0
+ return { id: f.id, name: f.name, ask: f.ask, type: f.type, found: got.length, of: rows.length, avgConfidence: Math.round(conf * 1000) / 1000, thin: rows.length > 2 && got.length / rows.length < 0.6 }
+ })
+ const findings = []
+ if (configError) findings.push({ severity: 'error', kind: 'job', ref: MARKER, message: configError })
+ if (jevError) findings.push({ severity: 'warning', kind: 'jev', message: jevError })
+ if (!chromePath) findings.push({ severity: 'error', kind: 'chrome', message: 'Google Chrome was not found on this machine. Install it, or set CHROME_PATH.' })
+ if (liveRoute() === null && rows.length) findings.push({ severity: 'warning', kind: 'jev', message: 'These rows came from the offline stand-in, not from Jev. They are not good enough to act on.' })
+ for (const f of filled) if (f.thin) findings.push({ severity: 'warning', kind: 'field', ref: f.id, message: `"${f.name}" was found on only ${f.found} of ${f.of} pages. Reword what it asks for, or the value may not be on those pages.` })
+ for (const e of (lastRun?.errors ?? []).slice(0, 3)) findings.push({ severity: 'warning', kind: 'page', ref: e.url, message: clean(e.message) })
+ return {
+ ready: phase === 'done' && rows.length > 0 && !configError,
+ summary: clean(configError ? `${MARKER} needs a fix: ${configError}`
+ : phase === 'idle' ? `Ready: ${job.task}. Press Start.`
+ : `${rows.length} collected from ${counters.pages} pages, ${counters.links} links judged, ${counters.calls} calls, $${counters.costUsd.toFixed(4)}${phase === 'running' ? ' (still going)' : ''}`).slice(0, 200),
+ findings, artifact: MARKER, resultsFile: RESULTS,
phases: [
- { id: 'site', name: 'Site up', state: 'done' },
- { id: 'book', name: 'Booking', state: world.steps > 0 || session.tasks > 0 ? 'active' : 'pending' },
- { id: 'score', name: 'Bookings checked', state: session.tasks > 0 ? 'done' : 'pending' },
+ { id: 'job', name: 'Job', state: configError ? 'failed' : 'done' },
+ { id: 'browse', name: 'Browsing', state: phase === 'running' ? 'active' : rows.length ? 'done' : 'pending' },
+ { id: 'review', name: 'Review', state: phase === 'done' ? (filled.some((f) => f.thin) ? 'active' : 'done') : 'pending' },
],
- })
+ run: {
+ client: liveRoute() ?? 'mock', task: job.task, start: job.start, item: job.item,
+ rows: rows.length, pages: counters.pages, linksJudged: counters.links, calls: counters.calls,
+ questions: counters.questions, costUsd: Math.round(counters.costUsd * 1e6) / 1e6,
+ elapsedMs: startedAt ? (finishedAt || Date.now()) - startedAt : 0, fields: filled,
+ kept: job.keep ? rows.filter((r) => r.keep).length : null,
+ },
+ }
}
+ let verdictTimer = null
+ const saveVerdict = () => { clearTimeout(verdictTimer); verdictTimer = null; try { writeVerdict(workspace, verdict()) } catch { /* gone */ } }
+ const verdictSoon = () => { if (!verdictTimer) verdictTimer = setTimeout(saveVerdict, 400) }
- function push(force = false) {
- const now = Date.now()
- if (force || now - lastVerdictAt > 1000) { lastVerdictAt = now; try { verdict() } catch (e) { error = clean(e.message) } }
- server?.broadcast(frame())
+ // ---- asking Jev --------------------------------------------------------------------------------
+ async function ask({ state, questions }) {
+ try {
+ const res = await evaluate({ state, questions, salt: 3, mock, model: process.env.JEV_MODEL || 'jev-latest' })
+ const n = Object.keys(questions).length
+ const real = Number(res.usage?.input_tokens)
+ counters.calls++
+ counters.questions += n
+ counters.tokens += Number.isFinite(real) && real > 0 ? real : Math.ceil((JSON.stringify(state).length + JSON.stringify(questions).length) / 4)
+ counters.costUsd = (counters.tokens * PRICE_PER_MTOK) / 1e6
+ jevError = null
+ return res
+ } catch (e) {
+ counters.errors++
+ jevError = clean(e?.message ?? e)
+ throw e
+ }
}
- function settle() {
- if (world.status !== 'done' || doneAt) return
- doneAt = Date.now()
- session.tasks++; if (world.result?.ok) session.ok++
- session.steps += world.steps; session.wasted += world.wasted; session.misclicks += world.misclicks
- session.wallMs += taskWallStart ? doneAt - taskWallStart : 0
- session.results.push({ goal: world.goal, ok: !!world.result?.ok, why: world.result?.why ?? '', steps: world.steps, wallMs: taskWallStart ? doneAt - taskWallStart : 0 })
- if (session.results.length > 60) session.results.shift()
+ // ---- the browser --------------------------------------------------------------------------------
+ async function openBrowser() {
+ if (chrome?.alive) return { open: true }
+ if (!chromePath) return { ok: false, error: 'Google Chrome was not found on this machine. Install it, or set CHROME_PATH to where it is.' }
+ const hosts = [...job.hosts]
+ if (demo) hosts.push('127.0.0.1', 'localhost')
+ chrome = await openChrome({
+ profileDir: join(workspace, '.harness', 'chrome-profile'),
+ show: show ?? (process.env.JEV_BROWSER_HEADLESS === '1' ? false : job.show),
+ allowedHosts: job.sameSiteOnly ? hosts : [],
+ })
+ say('The browser is open. Sign in to a site here if the pages you want need it.', 'browser')
+ startShots()
+ return { open: true }
+ }
+ async function closeBrowser() {
+ stopShots()
+ const c = chrome
+ chrome = null
+ await c?.close()
+ return { open: false }
}
+ function startShots() {
+ stopShots()
+ shotTimer = setInterval(async () => {
+ if (shotBusy || !chrome?.alive || !(server?.clients.size)) return
+ shotBusy = true
+ try { const s = await chrome.shot(); server?.broadcast({ ...s, url: here.url, at: Date.now() }, 'shot') } catch { /* between pages */ } finally { shotBusy = false }
+ }, 900)
+ }
+ function stopShots() { clearInterval(shotTimer); shotTimer = null }
- async function decide(humanId = null) {
- if (busy || stopped) return
- busy = true
- try {
- const c = cfg()
- if (world.status === 'done') {
- // Show the result for about 2.6 seconds of steps, then take the next task.
- world.rest = (world.rest ?? 0) + 1
- if (world.rest * c.stepMs >= 2600) { taskIndex++; if (taskIndex % world.taskCount === 0) round++; newTask() }
- return
- }
- if (!taskWallStart) taskWallStart = Date.now()
- const o = observe(world, c)
- stateText = o.text
- let choice = humanId
- if (!humanId) {
- const res = await evaluate({
- state: o.text,
- questions: {
- action: jev.choice(o.options, 'Which ONE element do you act on next to finish the task?'),
- done: jev.noul('Is the task already complete?'),
- obstacle: jev.noul('Is something covering the page that must be closed first?'),
+ // ---- the run --------------------------------------------------------------------------------------
+ async function start() {
+ if (running) return { ok: false, error: 'it is already running' }
+ if (configError) return { ok: false, error: configError }
+ // "demo" means the little made-up job board this harness serves itself.
+ if (/^demo$/i.test(job.start)) {
+ demo ??= await startDemoSite()
+ job = { ...job, start: demo.url, hosts: [...new Set([...job.hosts, '127.0.0.1', 'localhost'])] }
+ }
+ const opened = await openBrowser()
+ if (opened.ok === false) return opened
+ rows = []; links = []; feed = []
+ counters = { pages: 0, calls: 0, questions: 0, tokens: 0, costUsd: 0, links: 0, errors: 0 }
+ stopFlag = false; phase = 'running'; startedAt = Date.now(); finishedAt = 0
+ say(`Starting: ${job.task}`, 'start')
+ push(); saveVerdict()
+
+ running = (async () => {
+ try {
+ lastRun = await runJob({
+ chrome, job, ask, stopped: () => stopFlag,
+ onEvent: (e) => {
+ if (e.type === 'going') { here = { url: e.url, title: '' }; expecting = { expecting: e.what?.startsWith('list page') || e.url === job.start ? 'list' : 'item', item: job.item }; say(`Opening ${e.what || e.url}`, 'go') }
+ else if (e.type === 'read') { here = { url: e.url, title: e.title }; counters.pages++ }
+ else if (e.type === 'judged') { links = e.links ?? []; counters.links += e.judged; say(`${e.judged} links judged in one call: ${e.found} are ${job.item}${e.next ? `, next page "${e.next}"` : ''}`, 'judge') }
+ else if (e.type === 'row') { rows = [...rows, e.row]; say(`Collected #${e.row.n}: ${Object.values(e.row.fields).find(Boolean) ?? e.row.title}`, 'row'); saveResults() }
+ else if (e.type === 'skipped') say(`Nothing of the job on ${e.url}`, 'skip')
+ else if (e.type === 'trouble') { counters.errors++; say(`Trouble on ${e.url}: ${e.message}`, 'bad') }
+ else if (e.type === 'done') say(`Finished: ${e.rows} collected from ${e.pages} pages`, 'done')
+ push('view'); verdictSoon()
},
- salt: salt++, model: process.env.JEV_MODEL || 'jev-latest', mock: browserMock,
})
- const a = res.answers.action ?? {}
- choice = a.choice && o.options[a.choice] !== undefined ? a.choice : Object.keys(o.options)[0]
- last = { probs: a.probabilities ?? {}, choice, confidence: Number(a.confidence ?? 0), pDone: Number(res.answers.done?.noul ?? 0), pObstacle: Number(res.answers.obstacle?.noul ?? 0) }
- } else last = { ...last, choice: humanId, human: true }
- const rec = act(world, c, choice)
- if (rec && humanId) rec.note = rec.note ? `you clicked · ${rec.note}` : 'you clicked'
- stateText = observe(world, c).text
- settle()
- error = null
- } catch (e) {
- error = clean(e?.message ?? String(e))
- } finally {
- busy = false
- }
+ } catch (e) {
+ counters.errors++
+ say(e instanceof Refused ? `Refused: ${e.message}` : `Stopped: ${String(e.message ?? e)}`, 'bad')
+ } finally {
+ phase = stopFlag ? 'stopped' : 'done'
+ finishedAt = Date.now()
+ running = null
+ saveResults(); saveVerdict(); push()
+ }
+ })()
+ return { started: true }
}
- function schedule() { clearTimeout(timer); if (running && !stopped) timer = setTimeout(run, cfg().stepMs) }
- async function run() { const t0 = Date.now(); await decide(); push(); if (running && !stopped) timer = setTimeout(run, Math.max(0, cfg().stepMs - (Date.now() - t0))) }
+ async function stop() { stopFlag = true; say('Stopping after this page…', 'stop'); await running; return { phase } }
+ // ---- controls ---------------------------------------------------------------------------------
async function control(cmd, body) {
- if (cmd === 'pause') { running = false; clearTimeout(timer) }
- else if (cmd === 'start') { if (!running) { running = true; schedule() } }
- else if (cmd === 'reset') { Object.assign(session, { tasks: 0, ok: 0, steps: 0, wasted: 0, misclicks: 0, wallMs: 0, results: [] }); overrides = {}; taskIndex = 0; round = 0; salt = 1; newTask() }
- else if (cmd === 'tick') { const n = Math.round(clampN(body.n, 1, 20000, 1)); for (let i = 0; i < n; i++) await decide() }
- else if (cmd === 'click') { if (typeof body.id === 'string' && body.id.length < 40) await decide(body.id) }
- else if (cmd === 'popup') { if (!world.overlay && world.status === 'running') world.overlay = { kind: 'popup', at: [260, 150] }; stateText = observe(world, cfg()).text }
- else if (cmd === 'task') { taskIndex = Math.round(clampN(body.index, 0, 1000, 0)); newTask() }
- else if (cmd === 'set') {
- const allowed = { distraction: [0, 1], stepMs: [40, 2000] }
- if (allowed[body.key]) overrides = { ...overrides, [body.key]: clampN(body.value, allowed[body.key][0], allowed[body.key][1], cfg()[body.key]) }
+ switch (cmd) {
+ case 'start': return start()
+ case 'stop': return stop()
+ case 'run': { const r = await start(); await running; return { ...r, rows: rows.length, phase } } // used by tests: runs to the end
+ case 'openBrowser': return openBrowser()
+ case 'closeBrowser': return closeBrowser()
+ case 'reset': {
+ await stop()
+ rows = []; links = []; feed = []; phase = 'idle'; startedAt = 0; finishedAt = 0; lastRun = null
+ counters = { pages: 0, calls: 0, questions: 0, tokens: 0, costUsd: 0, links: 0, errors: 0 }
+ applyJob(watcher.get(), true); saveResults(); saveVerdict(); push()
+ return { phase }
+ }
+ case 'export': { saveResults(); return { file: RESULTS, rows: rows.length } }
+ case 'openHere': {
+ // The person asks the browser to go somewhere, from the pane. Same rules as everything else.
+ if (!chrome?.alive) { const o = await openBrowser(); if (o.ok === false) return o }
+ try { const r = await chrome.go(String(body.url ?? '')); here = { url: r.url, title: await chrome.title() }; push(); return { url: r.url } }
+ catch (e) { return { ok: false, error: e instanceof Refused ? e.message : String(e.message ?? e) } }
+ }
+ default: return { ok: false, error: `unknown command "${cmd}"` }
}
- push(true)
- return { steps: world.steps }
}
- newTask()
- server = await serveViewer({ here: HERE, port, state: frame, control })
- push(true)
- running = true
- schedule()
+ // ---- the job file --------------------------------------------------------------------------------
+ function applyJob(raw, fresh = false) {
+ const { job: next, errors } = normalizeJob(raw)
+ configError = errors.length ? `${MARKER}: ${errors.join('; ')}` : null
+ if (!configError || fresh || !job.fields.length) job = next
+ if (demo && /^demo$/i.test(job.start)) job = { ...job, start: demo.url, hosts: [...new Set([...job.hosts, '127.0.0.1', 'localhost'])] }
+ if (chrome) chrome.allowedHosts = job.sameSiteOnly ? [...job.hosts, ...(demo ? ['127.0.0.1', 'localhost'] : [])] : []
+ push(); verdictSoon()
+ }
+
+ mkdirSync(join(workspace, '.harness'), { recursive: true })
+ watcher = watchConfig(join(workspace, MARKER), {}, (cfg, err) => {
+ if (err) { configError = `${err}. Keeping the last good job`; push(); verdictSoon(); return }
+ applyJob(cfg)
+ })
+ server = await serveViewer({
+ here: HERE, port,
+ files: ['index.html', 'base.css', 'studio.css', 'studio.js', 'jev-hud.js'],
+ state: view, control,
+ downloads: () => { saveResults(); return { [RESULTS]: join(workspace, RESULTS) } },
+ onConnect: () => { jevError = null; push() },
+ })
+ applyJob(watcher.get(), true)
+ saveResults(); saveVerdict()
return {
url: server.url,
- async close() { stopped = true; running = false; clearTimeout(timer); cfgWatch.close(); await server.close() },
+ async close() {
+ stopFlag = true
+ await running
+ stopShots(); clearTimeout(verdictTimer)
+ watcher.close()
+ await closeBrowser()
+ await demo?.close()
+ await server.close()
+ },
}
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
const workspace = process.env.HARNESS_WORKSPACE
const port = Number(process.env.HARNESS_VIEWER_PORT)
- if (!workspace || !Number.isInteger(port) || port < 1 || port > 65535)
- throw new Error('HARNESS_WORKSPACE and HARNESS_VIEWER_PORT are required')
+ if (!workspace || !Number.isInteger(port) || port < 1 || port > 65535) throw new Error('HARNESS_WORKSPACE and HARNESS_VIEWER_PORT are required')
const viewer = await startBrowserViewer({ workspace, port })
console.log(`Jev Browser listening on ${viewer.url}`)
for (const s of ['SIGTERM', 'SIGINT']) process.once(s, () => viewer.close().then(() => process.exit(0)))
diff --git a/store/showcase/jev-browser/booked-exactly-right.jpg b/store/showcase/jev-browser/booked-exactly-right.jpg
deleted file mode 100644
index e9caa0835..000000000
Binary files a/store/showcase/jev-browser/booked-exactly-right.jpg and /dev/null differ
diff --git a/store/showcase/jev-browser/pick-the-flight.jpg b/store/showcase/jev-browser/pick-the-flight.jpg
deleted file mode 100644
index acc3ed007..000000000
Binary files a/store/showcase/jev-browser/pick-the-flight.jpg and /dev/null differ
diff --git a/store/showcase/jev-browser/reads-real-pages.jpg b/store/showcase/jev-browser/reads-real-pages.jpg
new file mode 100644
index 000000000..f36fa6546
Binary files /dev/null and b/store/showcase/jev-browser/reads-real-pages.jpg differ