Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

feat(jev-browser): give it a site and a sentence, and it finds its own way (#148)

Dee96146cbfd9d2
diff --git a/store/agents/jev-browser/AGENTS.md b/store/agents/jev-browser/AGENTS.md
index 7d423bbbd..d94dc3f38 100644
--- a/store/agents/jev-browser/AGENTS.md
+++ b/store/agents/jev-browser/AGENTS.md
@@ -27,8 +27,8 @@ do with Chrome yourself is slower, invisible to them, and doubles the work. Writ
let it run, and look at a page only if something comes back wrong.
1. **Take what they said and turn it into a job.** Often all you need is `start` and `want`:
- put their own sentence in `want` and leave `fields` out, and Jev reads the page and works the
- columns out in about a second. That is faster than you deciding, and it names the columns after
+ put their own sentence in `want` and leave `fields` out. `start` can be the whole site; Jev
+ walks it to the right page first, one call a step, then works the columns out in about a second. That is faster than you deciding, and it names the columns after
the page's own labels. Write the columns yourself only when they asked for something specific
that a page would not volunteer, such as a judgement ("is this remote?") or a score.
Do not interview them. Write something, let it run, then fix it.
@@ -51,7 +51,7 @@ let it run, and look at a page only if something comes back wrong.
```jsonc
{
"task": "Every flat for rent in the search results, with rent and address",
- "start": "https://example.com/search?area=leeds", // the page to begin on. "demo" is the made-up job board this harness serves itself
+ "start": "https://example.com", // where to begin: a site or the exact page. Jev walks to the right one. "demo" is the practice site this harness serves itself
"search": "", // optional: words to type into the site's own search box first
"want": "what each one costs and whether it is in stock", // the person's own words. Leave "fields" out and Jev works the columns out from the page
"item": "a flat for rent", // one of the things. Used in every question, so make it concrete
diff --git a/store/agents/jev-browser/README.md b/store/agents/jev-browser/README.md
index f7b7ab31e..6e1e5bf04 100644
--- a/store/agents/jev-browser/README.md
+++ b/store/agents/jev-browser/README.md
@@ -6,15 +6,20 @@ watch. No code, no selectors, no copy and paste.
## How you use it
-An address and a sentence is the whole job.
+A site and a sentence is the whole job.
| | |
|---|---|
-| **Start on** | `books.toscrape.com/catalogue/category/books/travel_2/index.html` |
-| **What you want** | `what each book costs and whether it is in stock` |
+| **Start on** | `books.toscrape.com` |
+| **What you want** | `travel books, with the price and whether they are in stock` |
-Press **Go**. Jev reads the page, works out what it lists and what the columns should be in about
-a second, and starts collecting. There is nothing else to press.
+Press **Go**. Jev walks the site to the page you meant, reads it, works out the columns, and
+starts collecting. There is nothing else to press.
+
+**It finds its own way.** The address does not have to be the list you want. Each step is one Jev
+call: does this page already show what was asked for, and if not, which link goes towards it? On
+that example it scores the "Travel" link at 0.97 out of the 62 links on the front page, follows it,
+and works from there. Give it the exact page if you have it and it starts there instead.
**Jev picks the columns, it does not write them.** It is shown every piece of text on one of the
pages and asked, for each, whether that is a fact about this thing worth a column. The column's
diff --git a/store/agents/jev-browser/skills/browser/SKILL.md b/store/agents/jev-browser/skills/browser/SKILL.md
index 3f103dec2..059efda37 100644
--- a/store/agents/jev-browser/skills/browser/SKILL.md
+++ b/store/agents/jev-browser/skills/browser/SKILL.md
@@ -22,7 +22,11 @@ a THING page → the details of one thing
A search page is a list page, so "search this site for X" is the same job: put the words in
`"search"` and the harness types them into the site's own search box, then reads the results.
-Point `start` at the list. If you point it at one thing's page, that one row is collected and the
+`start` does not have to be the list. Give it the site and Jev walks there: each step is one call
+asking whether this page is already the one, and if not which link goes towards it, up to five
+pages. Where a person gave you the exact page, use it and skip the walking.
+
+Point `start` at the list where you have it. If you point it at one thing's page, that one row is collected and the
run ends. If the things have no page of their own (everything is on the list), say so to the person:
this harness collects one row per page, so a list-only site gives one row for the whole list.
diff --git a/store/agents/jev-browser/template/browse.json b/store/agents/jev-browser/template/browse.json
index 620937074..707650df6 100644
--- a/store/agents/jev-browser/template/browse.json
+++ b/store/agents/jev-browser/template/browse.json
@@ -1,15 +1,7 @@
{
- "task": "Every open role on the job board, into a spreadsheet",
+ "task": "",
"start": "demo",
- "item": "a job posting",
- "fields": [
- { "id": "title", "name": "Job title", "ask": "the job title" },
- { "id": "company", "name": "Company", "ask": "the name of the company hiring" },
- { "id": "salary", "name": "Salary", "ask": "the pay or salary range" },
- { "id": "place", "name": "Location", "ask": "where the job is based" },
- { "id": "posted", "name": "Posted", "ask": "the date it was posted" },
- { "id": "remote", "name": "Remote?", "ask": "Can this job be done fully remotely?", "type": "yesno" }
- ],
+ "want": "what each one is called, what it pays and where it is",
"maxItems": 12,
"maxPages": 4,
"sameSiteOnly": true,
diff --git a/store/agents/jev-browser/test/viewer.test.mjs b/store/agents/jev-browser/test/viewer.test.mjs
index 3e4ab366f..4c551b2c4 100644
--- a/store/agents/jev-browser/test/viewer.test.mjs
+++ b/store/agents/jev-browser/test/viewer.test.mjs
@@ -7,7 +7,7 @@ import { tmpdir } from 'node:os'
import { connect } from 'node:net'
import { join, dirname } from 'node:path'
import { fileURLToPath } from 'node:url'
-import { normalizeJob, listQuestions, itemQuestions, rowFrom, pickLinks, proposeJob, THINGS, VALUE_KINDS, LIMITS } from '../viewer/crawl.mjs'
+import { normalizeJob, listQuestions, itemQuestions, rowFrom, pickLinks, proposeJob, findThePage, THINGS, VALUE_KINDS, LIMITS } from '../viewer/crawl.mjs'
import { readPage, blockOptions, blockText, canonical, pageState, wallReason, findSearchBox } from '../viewer/page.mjs'
import { openChrome, findChrome, checkUrl, Refused } from '../toolchain/chrome.mjs'
import { startDemoSite } from '../viewer/demosite.mjs'
@@ -16,6 +16,15 @@ import { startBrowserViewer } from '../viewer/viewer.mjs'
const HERE = dirname(fileURLToPath(import.meta.url))
const TEMPLATE = join(HERE, '../template')
const noChrome = findChrome() ? false : 'Google Chrome is not on this machine'
+// The starter job says nothing about jobs any more, so a test that wants columns brings its own.
+const FIELDS = [
+ { id: 'title', name: 'Job title', ask: 'the job title' },
+ { id: 'company', name: 'Company', ask: 'the name of the company hiring' },
+ { id: 'salary', name: 'Salary', ask: 'the pay or salary range' },
+ { id: 'place', name: 'Location', ask: 'where the job is based' },
+ { id: 'posted', name: 'Posted', ask: 'the date it was posted' },
+ { id: 'remote', name: 'Remote?', ask: 'Can this job be done fully remotely?', type: 'yesno' },
+]
process.env.JEV_BROWSER_HEADLESS = '1'
/** fetch will not send a foreign Host header, so the loopback guard is tested over a raw socket. */
@@ -31,11 +40,15 @@ const browser = async (allowedHosts = ['127.0.0.1', 'localhost']) =>
openChrome({ profileDir: mkdtempSync(join(tmpdir(), 'jev-browser-test-')), show: false, allowedHosts })
test('the job file: what it fills in, and what it says is wrong', () => {
+ // The starter job is an address and a sentence, and nothing about any particular kind of site.
const { job, errors } = normalizeJob(JSON.parse(readFileSync(join(TEMPLATE, 'browse.json'), 'utf8')))
assert.deepEqual(errors, [])
- assert.equal(job.fields.length, 6)
- assert.deepEqual(job.fields.map((f) => f.type), ['pick', 'pick', 'pick', 'pick', 'pick', 'yesno'])
+ assert.deepEqual(job.fields, [], 'the starter names no columns: Jev works them out')
+ assert.ok(job.want.length > 10, 'it says what is wanted, in a sentence')
assert.equal(job.sameSiteOnly, true)
+ const named = normalizeJob({ start: 'https://x.test/all', fields: FIELDS })
+ assert.deepEqual(named.errors, [])
+ assert.deepEqual(named.job.fields.map((f) => f.type), ['pick', 'pick', 'pick', 'pick', 'pick', 'yesno'])
const bare = normalizeJob({})
assert.match(bare.errors.join(' '), /no "start"/)
@@ -125,7 +138,7 @@ test('the questions: one call holds every link and every field', { skip: noChrom
const site = await startDemoSite()
const chrome = await browser()
try {
- const { job } = normalizeJob({ ...JSON.parse(readFileSync(join(TEMPLATE, 'browse.json'), 'utf8')), start: site.url })
+ const { job } = normalizeJob({ ...JSON.parse(readFileSync(join(TEMPLATE, 'browse.json'), 'utf8')), start: site.url, fields: FIELDS })
await chrome.go(site.url)
const list = await readPage(chrome)
const { questions, links } = listQuestions(list, job)
@@ -161,7 +174,7 @@ test('a whole run: rows land in results.csv, the verdict says what happened', {
const ws = mkdtempSync(join(tmpdir(), 'jev-browser-run-'))
cpSync(TEMPLATE, ws, { recursive: true })
const cfg = JSON.parse(readFileSync(join(ws, 'browse.json'), 'utf8'))
- writeFileSync(join(ws, 'browse.json'), JSON.stringify({ ...cfg, start: site.url, maxItems: 5, maxPages: 2 }))
+ writeFileSync(join(ws, 'browse.json'), JSON.stringify({ ...cfg, start: site.url, fields: FIELDS, maxItems: 5, maxPages: 2 }))
const v = await startBrowserViewer({ workspace: ws, port: 0 })
const ctl = async (cmd, body = {}) => (await fetch(`${v.url}/control`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ cmd, ...body }) })).json()
try {
@@ -416,3 +429,37 @@ test("a proposed column is named off the page, and none of it is written", { ski
}
} finally { await chrome.close(); await site.close(); delete process.env.JEV_OFFLINE }
})
+
+test('a whole site is enough: Jev walks to the page the person meant', { skip: noChrome }, async () => {
+ // Typing "the site" used to do nothing sensible, which made the tool look like it only knew
+ // about one kind of page. It now walks: is this it, and if not, which link goes towards it.
+ process.env.JEV_OFFLINE = '1'
+ const site = await startDemoSite()
+ const chrome = await browser()
+ try {
+ const { evaluate } = await import('../toolchain/jev.mjs')
+ const ask = (x) => evaluate({ ...x, salt: 3 })
+ const steps = []
+ const got = await findThePage({ chrome, ask, start: new URL('/about', site.url).toString(), want: 'the open roles and what they pay', onEvent: (e) => steps.push(e.type) })
+ assert.equal(got.walled, undefined)
+ assert.ok(got.steps >= 1 && got.steps <= 5)
+ assert.ok(steps.includes('going'))
+ assert.ok(got.page, 'it hands back the page it stopped on, so nothing is read twice')
+ assert.equal(got.page.url, got.url)
+
+ // A page that is already the list is recognised without wandering off it.
+ const there = await findThePage({ chrome, ask, start: site.url, want: 'the open roles and what they pay' })
+ assert.equal(there.steps, 1, 'it stops where it starts when that is the page')
+ assert.equal(there.gaveUp, undefined)
+ assert.ok(there.url.startsWith(site.url))
+
+ // A site that will not be read is reported, not walked round.
+ const { createServer } = await import('node:http')
+ const blocker = createServer((_q, r) => { r.writeHead(200, { 'content-type': 'text/html' }); r.end('<title>Just a moment...</title><body>Checking your browser</body>') })
+ await new Promise((r) => blocker.listen(0, '127.0.0.1', r))
+ const walled = await findThePage({ chrome, ask, start: `http://127.0.0.1:${blocker.address().port}/`, want: 'anything' })
+ blocker.close()
+ assert.match(walled.walled, /block page|nearly empty/)
+ assert.equal(walled.steps, 1, 'it stops at the wall rather than trying more pages')
+ } finally { await chrome.close(); await site.close(); delete process.env.JEV_OFFLINE }
+})
diff --git a/store/agents/jev-browser/viewer/crawl.mjs b/store/agents/jev-browser/viewer/crawl.mjs
index 04320b52f..39d78ee54 100644
--- a/store/agents/jev-browser/viewer/crawl.mjs
+++ b/store/agents/jev-browser/viewer/crawl.mjs
@@ -121,6 +121,69 @@ export function pickLinks(links, answers, { threshold = 0.5 } = {}) {
}
+
+// ---- finding the page to work from ---------------------------------------------------------------
+/**
+ * Walk a site until the page in front of us is the one the person asked for. Every step is one Jev
+ * call: is this it, and if not, which way. Nothing is clicked; links are followed by their address,
+ * and a search box is used only when the site's own search is the obvious way in.
+ * @returns {Promise<{ url, page, steps, walled, gaveUp }>}
+ */
+export async function findThePage({ chrome, ask, start, want, search = '', maxSteps = 5, onEvent, stopped = () => false }) {
+ const say = (type, data) => { try { onEvent?.({ type, at: Date.now(), ...data }) } catch { /* the pane may be gone */ } }
+ let url = start, page = null, searched = false
+ for (let step = 0; step < maxSteps && !stopped(); step++) {
+ say('going', { url, what: step === 0 ? 'the page you gave' : 'looking for the right page' })
+ await chrome.go(url)
+ page = await readPage(chrome)
+ const wall = wallReason(page)
+ if (wall) return { url, page, steps: step + 1, walled: `${new URL(page.url).hostname}: ${wall}` }
+
+ const links = page.links.slice(0, LIMITS.linksPerCall)
+ const box = findSearchBox(page)
+ const asked = want || 'what the person asked for'
+ const q = {
+ here: jev.noul(`The person asked for: "${asked}". Does THIS page already show a list of those, with a link to each one?`,
+ { true: 'this page lists them', false: 'this page is a front page, a menu, an article, or about something else' }),
+ one: jev.noul(`The person asked for: "${asked}". Is THIS page one single one of those, in detail, rather than a list of them?`),
+ }
+ for (const l of links) {
+ q[`g${l.n}`] = jev.noul(`The person asked for: "${asked}". This page has a link reading "${l.label}" going to ${l.path}. Would following it get closer to what they asked for?`,
+ { true: 'it leads towards what they asked for', false: 'it leads away: an unrelated section, a login, a legal page, or an advert' })
+ }
+ if (box && !searched && search) q.usesearch = jev.noul(`This page has a search box. The person asked for: "${asked}". Is searching this site the best way to find them, rather than following a link?`)
+ const res = await ask({ state: pageState(page, { what_they_want: asked }), questions: q })
+
+ const here = res.answers.here?.noul ?? 0
+ const one = res.answers.one?.noul ?? 0
+ const ranked = links.map((l) => ({ l, p: res.answers[`g${l.n}`]?.noul ?? 0 })).sort((a, b) => b.p - a.p)
+ const best = ranked[0]
+ say('judged', { url: page.url, judged: links.length, found: 0, latencyMs: res.latencyMs,
+ links: ranked.map(({ l, p }) => ({ n: l.n, label: l.label, p })).slice(0, 40) })
+ // A front page of a bookshop IS a list of books, so "does this page show them" says yes and the
+ // walk stops one click short of the travel section. A link the person's own words point
+ // straight at beats a page that merely qualifies.
+ const obvious = best && best.p >= 0.75 && here < 0.85
+ if ((here >= 0.5 && !obvious) || one >= 0.6) { say('arrived', { url: page.url, listing: here >= 0.5, steps: step + 1 }); return { url: page.url, page, steps: step + 1, isOne: one >= 0.6 && here < 0.5 } }
+
+ if (search && box && !searched && (res.answers.usesearch?.noul ?? 0) >= 0.5) {
+ say('searching', { words: search })
+ searched = true
+ await chrome.search(box.css, search)
+ url = await chrome.url()
+ continue
+ }
+ if (!best || best.p < 0.5) {
+ // Nothing points onward. If this page at least shows the right sort of thing, work from it.
+ if (here >= 0.5) { say('arrived', { url: page.url, listing: true, steps: step + 1 }); return { url: page.url, page, steps: step + 1 } }
+ return { url: page.url, page, steps: step + 1, gaveUp: 'no link on this page looked like the way to it' }
+ }
+ say('step', { label: best.l.label, p: best.p })
+ url = best.l.url
+ }
+ return { url, page, steps: maxSteps, gaveUp: `it was still looking after ${maxSteps} pages` }
+}
+
// ---- proposing a job ----------------------------------------------------------------------------
/** What a page can be listing. A fixed list, so Jev only has to point at one. */
export const THINGS = {
@@ -153,12 +216,11 @@ export const VALUE_KINDS = {
* @param {string} [o.search]
* @returns {Promise<{ kind, item, columns, sample, from, walled }>}
*/
-export async function proposeJob({ chrome, ask, start, want = '', search = '', onEvent }) {
+export async function proposeJob({ chrome, ask, start, want = '', search = '', page = null, onEvent }) {
const say = (type, data) => { try { onEvent?.({ type, at: Date.now(), ...data }) } catch { /* the pane may be gone */ } }
- say('going', { url: start, what: 'the page you gave' })
- await chrome.go(start)
- let list = await readPage(chrome)
- if (search) {
+ let list = page
+ if (!list) { say('going', { url: start, what: 'the page you gave' }); await chrome.go(start); list = await readPage(chrome) }
+ if (search && !page) {
const box = findSearchBox(list)
if (box) { say('searching', { words: search }); await chrome.search(box.css, search); list = await readPage(chrome) }
}
diff --git a/store/agents/jev-browser/viewer/index.html b/store/agents/jev-browser/viewer/index.html
index ef3990d5b..b170990bb 100644
--- a/store/agents/jev-browser/viewer/index.html
+++ b/store/agents/jev-browser/viewer/index.html
@@ -45,15 +45,15 @@
<img id="screen" alt="">
<div id="screenIdle" class="screen-idle">
<form id="askForm" class="ask">
- <div class="ask-head"><b>What do you want off the web?</b><span>An address and a sentence is enough. Jev reads the page, works out the columns in about a second, and goes.</span></div>
+ <div class="ask-head"><b>What do you want off the web?</b><span>A site and a sentence is enough. Jev finds the right page, works out the columns in about a second, and goes.</span></div>
<label class="ask-row"><span>Start on</span>
- <input id="askStart" type="text" spellcheck="false" autocomplete="off" placeholder="books.toscrape.com/catalogue/category/books/travel_2/index.html">
+ <input id="askStart" type="text" spellcheck="false" autocomplete="off" placeholder="books.toscrape.com &nbsp;— a site or the exact page">
</label>
<label class="ask-row"><span>Search for <i>optional</i></span>
<input id="askSearch" type="text" spellcheck="false" autocomplete="off" placeholder="leave empty if the address is already the list you want">
</label>
<label class="ask-row"><span>What you want</span>
- <input id="askWant" type="text" spellcheck="false" autocomplete="off" placeholder="what each book costs and whether it is in stock">
+ <input id="askWant" type="text" spellcheck="false" autocomplete="off" placeholder="travel books, with the price and whether they are in stock">
</label>
<label class="ask-row"><span>Each one is <i>optional</i></span>
<input id="askItem" type="text" spellcheck="false" autocomplete="off" placeholder="Jev works this out from the page">
diff --git a/store/agents/jev-browser/viewer/viewer.mjs b/store/agents/jev-browser/viewer/viewer.mjs
index 9287c578d..f95bd031a 100644
--- a/store/agents/jev-browser/viewer/viewer.mjs
+++ b/store/agents/jev-browser/viewer/viewer.mjs
@@ -9,7 +9,7 @@ import { writeFileSync, renameSync, mkdirSync } from 'node:fs'
import { evaluate, PRICE_PER_MTOK, resolveCredentials, describeCredentials } from '../toolchain/jev.mjs'
import { serveViewer, writeVerdict, watchConfig, clean } from './kit.mjs'
import { openChrome, findChrome, Refused } from '../toolchain/chrome.mjs'
-import { normalizeJob, runJob, proposeJob, LIMITS } from './crawl.mjs'
+import { normalizeJob, runJob, proposeJob, findThePage, LIMITS } from './crawl.mjs'
import { browserMock } from './mock.mjs'
import { startDemoSite } from './demosite.mjs'
@@ -174,13 +174,28 @@ export async function startBrowserViewer({ workspace, port = 0, show = null } =
// second, and the answer is written back into browse.json so the job stays the recipe.
if (!job.fields.length) {
proposing = true
- say('Reading the page to work out the columns…', 'start')
+ say('Looking for the right page…', 'start')
counters = { pages: 0, calls: 0, questions: 0, tokens: 0, costUsd: 0, links: 0, errors: 0 }
startedAt = Date.now(); finishedAt = 0; phase = 'running'
push()
+ const watch = (e) => {
+ if (e.type === 'judged') links = e.links ?? []
+ if (e.type === 'going') here = { url: e.url, title: '' }
+ if (e.type === 'step') say(`Following "${e.label}" (${Math.round(e.p * 100)}%)`, 'go')
+ if (e.type === 'searching') say(`Searching the site for "${e.words}"`, 'go')
+ if (e.type === 'arrived') say(`This is the page: ${e.url}`, 'judge')
+ push('view')
+ }
let got
try {
- got = await proposeJob({ chrome, ask, start: job.start, want: job.want, search: job.search, onEvent: (e) => { if (e.type === 'judged') links = e.links ?? []; if (e.type === 'going') here = { url: e.url, title: '' }; push('view') } })
+ // The address may be a whole site rather than the list itself. Walk to it first.
+ const found = await findThePage({ chrome, ask, start: job.start, want: job.want, search: job.search, onEvent: watch, stopped: () => stopFlag })
+ if (found.walled) { got = { walled: found.walled, columns: [] } }
+ else {
+ if (found.gaveUp) say(`Still looking: ${found.gaveUp}. Reading this page anyway.`, 'bad')
+ if (found.url !== job.start) { job = { ...job, start: found.url }; counters.pages = found.steps }
+ got = await proposeJob({ chrome, ask, start: found.url, page: found.page, want: job.want, search: job.search, onEvent: watch })
+ }
} catch (e) { got = { columns: [], error: String(e.message ?? e) } }
proposing = false
if (got.walled) { lastRun = { walled: got.walled, errors: [] }; say(got.walled, 'bad'); phase = 'done'; finishedAt = Date.now(); push(); saveVerdict(); return { ok: false, error: got.walled } }
@@ -192,7 +207,9 @@ export async function startBrowserViewer({ workspace, port = 0, show = null } =
return { ok: false, error: `${why}. Write the columns yourself, one per line.` }
}
const saidItem = String(watcher.get()?.item ?? '').trim()
- const next = { ...watcher.get(), item: saidItem || got.item, fields: got.columns.map((c) => ({ id: c.id, name: c.name, ask: c.ask })) }
+ // Write back the page it walked to, not the address that was typed: otherwise the job resets
+ // to the site's front page and collects whatever happens to be on it.
+ const next = { ...watcher.get(), start: job.start, item: saidItem || got.item, fields: got.columns.map((c) => ({ id: c.id, name: c.name, ask: c.ask })) }
writeMarker(next)
applyJobQuiet(next)
say(`Jev found ${got.columns.length} columns in ${got.ms} ms: ${got.columns.map((c) => c.name).slice(0, 6).join(', ')}`, 'judge')