Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Fail a page read when a wall's text changes on every read

Claudee445fe83b329
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2218bbf70..3dfb1f80e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,7 +8,7 @@ This changelog was generated from the repository Git history and release tags. V
### Fixed
- Chat sites such as WhatsApp: a request that points at the open chat instead of naming the person, such as "send some test messages to this one" or "message this user", now sends there instead of asking "which contact, and what text?". The planner never sees the page, and its rules counted only words like "reply here" as the open chat, so it asked the same question every time without reading the page. WebBrain still reads the open chat's header itself and binds every send to that one contact, and stops if no single chat is open. The planner is also told it cannot see the page, no longer reads a note about a screenshot it is never given, and leaves the wording of described messages, such as test messages or a thank-you, to the agent. One request can now cover the open chat and someone named besides it, as in "test messages here, then thank Rustem Bey": the open chat needs no confirmation, and the named person is confirmed once on WebBrain's card before the first send to them.
-- A subscription, registration or sign-in wall in front of a news article is now recognized in any language, by a model reading the wall's text once, instead of only when it used English phrases such as "Subscribe to continue reading". A Turkish "Okumaya devam etmek için abone olun" overlay now keeps WebBrain from reading the article hidden behind it, as an English one did, while an offer to subscribe beside text you can still read is not taken for a wall. A wall inside the article is found also after a long preview, and everything after the wall's text stays withheld; the preview before it stays readable, except the short passage just before the wall's text. If a page cannot be read again after its wall was recognized, the read fails rather than return what the wall hides.
+- A subscription, registration or sign-in wall in front of a news article is now recognized in any language, by a model reading the wall's text once, instead of only when it used English phrases such as "Subscribe to continue reading". A Turkish "Okumaya devam etmek için abone olun" overlay now keeps WebBrain from reading the article hidden behind it, as an English one did, while an offer to subscribe beside text you can still read is not taken for a wall. A wall inside the article is found also after a long preview, and everything after the wall's text stays withheld; the preview before it stays readable, except the short passage just before the wall's text. If a page cannot be read again after its wall was recognized, or its wall's text changes on every read, as with a countdown, the read fails rather than return what the wall hides.
- A dialog asking you to prove you are human now pauses the run's page actions in any language, not only when it says "Security verification", "Verify you are human" or another English phrase. Its title is read by a model once, the decision model first; LinkedIn's Turkish "Güvenlik doğrulaması" now counts, while a cookie notice or a form dialog does not. A dialog naming a CAPTCHA product still counts without any reading. A title no model can answer stops nothing and is asked about again only after half a minute, so page actions do not each wait for it.
- Finding a verification email: the service the code is for is now matched by its name or web address alone, with no English words such as "code" or "verification" set aside, so the search treats the same request in every language alike. The tool now asks the model for only the service's name or domain.
- An AWS WAF puzzle page ("Let's confirm you are human" in English) is now recognized in any language, by AWS's own puzzle script on its interstitial page, instead of by its English heading. Before, the same page in another language was not recognized as a CAPTCHA unless its widget was found, and a page running only AWS's silent check is still not taken for a puzzle.
diff --git a/docs/architecture.md b/docs/architecture.md
index 1eb2d222d..f2fc92152 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -600,7 +600,9 @@ a link standing on its own), in `pageTextsNeeded`; the agent has them read
(the decision model first, the active model within four seconds) and reads
the page again with the readings, up to three times while each read reaches
more. When a text reads as a wall and the page cannot be read again, the
-read fails rather than return the answer made without that reading. An
+read fails rather than return the answer made without that reading; so does
+a last read that still reports texts after a wall was read but not applied,
+as with a wall whose text changes on every read, such as a countdown. An
unread text is no wall, so the page reads as it is, and a text no model
answered after both had their turn is not asked again for 30 seconds. A wall
withholds the article, its links, forms and frames; its own text and
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 27434c495..7bd7b3ca5 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -12196,16 +12196,20 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// most forty at a time, the likeliest walls first). A read that cannot be
// repeated keeps the answer before it, unless one of the texts that answer
// was made without reads as a wall: that answer may hold what the wall
- // withholds, so the read fails instead.
+ // withholds, so the read fails instead. So does a last answer that still
+ // waits on texts after a wall was read but not applied, as with a wall
+ // whose text changes on every read, such as a countdown.
async _rereadWithPageBlockerReadings(tabId, name, response, dispatch) {
if ((name !== 'read_page' && name !== 'get_accessibility_tree') || !response || typeof response !== 'object') return response;
let current = response;
+ let wallReadLast = false;
for (let pass = 0; pass < 3; pass += 1) {
const needed = current.pageTextsNeeded;
delete current.pageTextsNeeded;
if (!await this._readPageBlockerTexts(tabId, needed)) return current;
const wallRead = (Array.isArray(needed) ? needed : []).some(entry => entry?.kind === 'page_blocker'
&& ['subscription', 'registration', 'login'].includes(this._pageTextReading('page_blocker', entry.text)));
+ wallReadLast = wallRead;
const failed = () => (wallRead ? { success: false, error: 'This page shows a subscription, registration or '
+ 'sign-in wall, and reading the page again with it failed, so nothing behind the wall was returned. '
+ 'Read the page again.' } : current);
@@ -12219,7 +12223,13 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (!reread || typeof reread !== 'object' || reread.success === false) return failed();
current = reread;
}
+ const stillUnread = Array.isArray(current.pageTextsNeeded) && current.pageTextsNeeded.length > 0;
delete current.pageTextsNeeded;
+ if (wallReadLast && stillUnread && !current.pageGate) {
+ return { success: false, error: 'This page shows a subscription, registration or sign-in wall whose text '
+ + 'changes from one read to the next, so WebBrain could not apply it, and nothing behind it was returned. '
+ + 'Report the wall instead of reading past it.' };
+ }
return current;
}
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 308f227db..30e495e56 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -10829,16 +10829,20 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// most forty at a time, the likeliest walls first). A read that cannot be
// repeated keeps the answer before it, unless one of the texts that answer
// was made without reads as a wall: that answer may hold what the wall
- // withholds, so the read fails instead.
+ // withholds, so the read fails instead. So does a last answer that still
+ // waits on texts after a wall was read but not applied, as with a wall
+ // whose text changes on every read, such as a countdown.
async _rereadWithPageBlockerReadings(tabId, name, response, dispatch) {
if ((name !== 'read_page' && name !== 'get_accessibility_tree') || !response || typeof response !== 'object') return response;
let current = response;
+ let wallReadLast = false;
for (let pass = 0; pass < 3; pass += 1) {
const needed = current.pageTextsNeeded;
delete current.pageTextsNeeded;
if (!await this._readPageBlockerTexts(tabId, needed)) return current;
const wallRead = (Array.isArray(needed) ? needed : []).some(entry => entry?.kind === 'page_blocker'
&& ['subscription', 'registration', 'login'].includes(this._pageTextReading('page_blocker', entry.text)));
+ wallReadLast = wallRead;
const failed = () => (wallRead ? { success: false, error: 'This page shows a subscription, registration or '
+ 'sign-in wall, and reading the page again with it failed, so nothing behind the wall was returned. '
+ 'Read the page again.' } : current);
@@ -10852,7 +10856,13 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (!reread || typeof reread !== 'object' || reread.success === false) return failed();
current = reread;
}
+ const stillUnread = Array.isArray(current.pageTextsNeeded) && current.pageTextsNeeded.length > 0;
delete current.pageTextsNeeded;
+ if (wallReadLast && stillUnread && !current.pageGate) {
+ return { success: false, error: 'This page shows a subscription, registration or sign-in wall whose text '
+ + 'changes from one read to the next, so WebBrain could not apply it, and nothing behind it was returned. '
+ + 'Report the wall instead of reading past it.' };
+ }
return current;
}
diff --git a/test/run.js b/test/run.js
index a1acbc55f..b1a104c95 100644
--- a/test/run.js
+++ b/test/run.js
@@ -18562,6 +18562,45 @@ test('a page read reaching more unread wall texts after a reread is read again,
}
});
+// From review: a wall whose text changes on every read, such as a countdown,
+// was read as a wall each time, yet the page never met that reading again,
+// so after three reads the article behind it was returned.
+test('a wall whose text changes on every read fails the read instead of returning what it hides', async () => {
+ for (const [build, AgentClass] of [['chrome', AgentCh], ['firefox', AgentFx]]) {
+ const agent = new AgentClass({});
+ stubPageTextReadings(agent, (kind, item) => (kind === 'page_blocker'
+ ? (/abone olun/.test(item.text) ? 'subscription' : 'other') : null));
+ const wallAt = n => [{ kind: 'page_blocker', text: `Okumaya devam etmek için abone olun. Kalan süre: ${59 - n} sn` }];
+ let reads = 0;
+ const result = await agent._rereadWithPageBlockerReadings(1, 'read_page',
+ { text: 'SECRET_0', pageTextsNeeded: wallAt(0) },
+ async () => { reads += 1; return { text: `SECRET_${reads}`, pageTextsNeeded: wallAt(reads) }; });
+ assert.equal(reads, 3, `${build}: the page was not read again with each new wall text`);
+ assert.equal(result.success, false, `${build}: the article behind a changing wall was returned`);
+ assert.doesNotMatch(JSON.stringify(result), /SECRET/, `${build}: text behind the wall leaked`);
+ assert.match(result.error, /wall whose text changes from one read to the next/);
+ // The same wall applied on the last read is returned as the page shows it.
+ const applied = new AgentClass({});
+ stubPageTextReadings(applied, (kind, item) => (kind === 'page_blocker'
+ ? (/abone olun/.test(item.text) ? 'subscription' : 'other') : null));
+ let appliedReads = 0;
+ const gated = await applied._rereadWithPageBlockerReadings(1, 'read_page',
+ { text: 'SECRET_0', pageTextsNeeded: wallAt(0) },
+ async () => { appliedReads += 1; return appliedReads < 3
+ ? { text: `SECRET_${appliedReads}`, pageTextsNeeded: wallAt(appliedReads) }
+ : { text: 'gate', pageGate: { type: 'subscription' }, pageTextsNeeded: [{ kind: 'page_blocker', text: 'Bülten' }] }; });
+ assert.deepEqual(gated, { text: 'gate', pageGate: { type: 'subscription' } }, `${build}: an applied wall was refused`);
+ // Texts read as no wall leave the last read as it is.
+ const plain = new AgentClass({});
+ stubPageTextReadings(plain, () => 'other');
+ let plainReads = 0;
+ const last = await plain._rereadWithPageBlockerReadings(1, 'read_page',
+ { text: 'article 0', pageTextsNeeded: wallAt(0) },
+ async () => { plainReads += 1; return { text: `article ${plainReads}`, pageTextsNeeded: wallAt(plainReads) }; });
+ assert.deepEqual(last, { text: 'article 3' }, `${build}: a page with no wall was refused`);
+ }
+});
+
test('a wall or dialog title no model answered is not asked again on every read for a while', async () => {
for (const [build, AgentClass] of [['chrome', AgentCh], ['firefox', AgentFx]]) {
const realNow = Date.now;