Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Save the attempt record with the conversation; review fixes

Claudeda4d53e7e57d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 24d4c99fd..2e03af1f8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,7 +7,7 @@ This changelog was generated from the repository Git history and release tags. V
## [Unreleased]
### Fixed
-- Posting on another site after a correction: after a Mastodon post, "ok now on X" no longer leaves WebBrain going back and forth between the two sites, typing the X post again each time. It counted X as the same platform as Mastodon, because it reads account names only on X and Bluesky, and asked to find the Mastodon post on X itself, which can never happen. The site a post was made on now names its platform when no account name was read, and a post on another site goes ahead once a page has been read since the correction, provided the task made no earlier attempt on that site. Each correction now records every site the task had attempted, so this also holds when WebBrain restarted in between. When the task did attempt that site before, or WebBrain has no full record of its earlier attempts, it stops once and says so, instead of searching for a post it cannot find. A search, or a click that sent nothing, is not an attempt, and x.com and twitter.com are one platform.
+- Posting on another site after a correction: after a Mastodon post, "ok now on X" no longer leaves WebBrain going back and forth between the two sites, typing the X post again each time. It counted X as the same platform as Mastodon, because it reads account names only on X and Bluesky, and asked to find the Mastodon post on X itself, which can never happen. The site a post was made on now names its platform when no account name was read, and a post on another site goes ahead once a page has been read since the correction, provided the task made no earlier attempt on that site. What the task attempted is saved with the conversation, so this still holds when the task continues after WebBrain restarted, and a post already sent on X under the correction is not sent again by a later run of the task. When the task did attempt that site before, or WebBrain has no full record of its earlier attempts, it stops once and says so, instead of searching for a post it cannot find. A search, a click that sent nothing or a form the page rejected is not an attempt; a click that Stop interrupted after it was sent is. x.com and twitter.com are one platform.
- Chrome: a page's "Leave site?" prompt no longer waits for you when WebBrain navigates away during a task that started on a page the debugger cannot attach to, such as chrome://extensions, or whose debugger connection dropped. WebBrain arms its dialog handling again at the next debugger attach and around each of its own navigations, and answers Leave for its own navigation as it already did on other runs. If you cancel the debugger, prompts stay yours to answer. Firefox offers extensions no way to answer these prompts.
- Chat sites such as WhatsApp: a request that points at the open chat instead of naming the person, such as "send some test messages to this one" or "message this user", now sends there instead of asking "which contact, and what text?". The planner never sees the page, and its rules counted only words like "reply here" as the open chat, so it asked the same question every time without reading the page. WebBrain still reads the open chat's header itself and binds every send to that one contact, and stops if no single chat is open. The planner is also told it cannot see the page, no longer reads a note about a screenshot it is never given, and leaves the wording of described messages, such as test messages or a thank-you, to the agent. One request can now cover the open chat and someone named besides it, as in "test messages here, then thank Rustem Bey": the open chat needs no confirmation, the reviewed plan shows the named person, and that person is confirmed once on WebBrain's card before the first send to them. Words that refer back to someone named earlier still mean that person, and on mail sites the people named besides "reply here" are handled as before. On WhatsApp, where a contact is confirmed through its contact drawer, one run still confirms only one chat's contact, so the second chat of such a request needs a request of its own; the agent is now told so, instead of being sent to re-read the page and retry.
- A subscription, registration or sign-in wall in front of a news article is now recognized in any language, by a model reading the wall's text once, instead of only when it used English phrases such as "Subscribe to continue reading". A Turkish "Okumaya devam etmek için abone olun" overlay now keeps WebBrain from reading the article hidden behind it, as an English one did, while an offer to subscribe beside text you can still read is not taken for a wall. A wall inside the article is found also after a long preview, and everything after the wall's text stays withheld; the preview before it stays readable, except the short passage just before the wall's text. If a page cannot be read again after its wall was recognized, the read fails rather than return what the wall hides. A wall whose number changes on every read, such as a countdown, stays a wall while its new text is read, and needs no more readings once two of its numbers were read as the wall, while a count that stops being a wall when its number changes reads as what it is once read; a wall whose other text keeps changing makes the read fail rather than leak, and the agent may read the page once more in case the wall went away.
diff --git a/docs/architecture.md b/docs/architecture.md
index 92b13ee89..0d9fe052d 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -103,15 +103,22 @@ Mastodon post or a form, is placed by the site it was made on. Hosts that share
their last two labels count as one site, and x.com and twitter.com as one
platform. A post on another site than the earlier attempt counts as another
platform only when the correction's receipt lists every site the task had
-attempted and the post's site is not among them. The agent records the site of
-every dispatched attempt that may have changed something (a search, or a send
-the browser refused, is not one) for the task, from its first run on; each
-correction copies that list into its receipt, and a run rebuilt from the saved
-conversation, for example after a restart, reads it there. Without a complete
-list, or with an earlier attempt on the post's own site, the post stops once
-with a report instead of a search for the other site's post. For a post on
-another platform, any page read since the earlier attempt was sent, including
-one after the correction, makes its outcome known. Whether a correction
+attempted and the post's site is not among them. The agent records, for the
+task from its first run on, the site of every dispatched attempt that may have
+changed something and of every publication among them (a search, a send the
+browser refused or a form the page rejected is not one; an action Stop
+interrupted after it was sent is). The record is saved with the conversation,
+and each correction copies its sites into its receipt. A post made under the
+correction is kept from repeating by the publication contract's outcomes and
+dispatch records of the run that applied it; a run rebuilt from the saved
+conversation, as after a restart or a typed "continue", has neither, so there
+a publication anywhere in the task on the post's site holds it too. Without a
+complete record, or with an earlier attempt on the post's own site, the post
+stops once with a report instead of a search for the other site's post. For
+such a post, any page read since the earlier attempt was sent, including one
+after the correction, makes its outcome known; where account names or an
+ordinary form tell the targets apart, the earlier result must be observed as
+before. Whether a correction
asks to delete the earlier post or record first is a fixed-choice question
that a model answers from the correction as the user wrote it, in any language
(see "Language decisions" below): removing characters, words or hashtags from a
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 135442b03..1dd3cd90a 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -2120,6 +2120,8 @@ export class Agent extends LoopDetector {
actionRef: name !== 'iframe_click' && /^ref_\d+$/.test(String(args?.ref_id || '')) ? args.ref_id : '',
actionSequence: Number(this.completionInvariants.get(tabId)?.lastAction?.sequence || 0),
submitLike: true,
+ // A search or an in-form toggle reaches here too; it commits nothing.
+ ...(this._nonCommittingSubmission(detectedSubmit) ? { nonCommitting: true } : {}),
semanticCommit: semanticCommit ? {
taskKey: this._progressTaskKeyHash(tabId), runToken: this.completionInvariants.get(tabId)?.runToken,
sourceSha256: this._sha256TextSync(this._progressTaskAnchorText(tabId)),
@@ -5507,6 +5509,9 @@ export class Agent extends LoopDetector {
guard.steeringPriorSubmission = priorSubmission ? { ...structuredClone(priorSubmission),
taskAttemptPlatforms: this._correctionAttemptPlatforms(tabId, messages, this._activeTaskBinding(messages), priorSubmission) } : null;
guard.steeringPriorReadAfterCorrection = false;
+ // This run applied the correction: what it sends under it stays in its
+ // publication contract and dispatch records.
+ guard.steeringHistoryInMemory = true;
guard.steeringMutationReceipts = structuredClone(previous?.steeringMutationReceipts || []);
// Action sequences restart with each run, so a receipt is one attempt
// only within its run.
@@ -5631,13 +5636,13 @@ export class Agent extends LoopDetector {
// Outcome knowledge that does not depend on the current page: a cached
// reconciliation of this exact receipt, a job-verified publication outcome,
- // or an explicit page observation after the dispatch, including any page
- // read under the correction, which came after it.
- _steeringPriorEffectKnown(tabId, guard, prior) {
+ // or an explicit page observation after the dispatch. Where the caller asks,
+ // any page read under the correction counts too, since it came after it.
+ _steeringPriorEffectKnown(tabId, guard, prior, { readAfterCorrection = false } = {}) {
const cached = guard?.steeringPriorSubmissionReconciliation;
return (cached?.verified === true && cached.actionSequence === prior?.actionSequence)
|| this._steeringPriorPublicationOutcome(guard, prior) === 'verified'
- || (!!prior && guard?.steeringPriorSubmission === prior && guard.steeringPriorReadAfterCorrection === true)
+ || (readAfterCorrection && !!prior && guard?.steeringPriorSubmission === prior && guard.steeringPriorReadAfterCorrection === true)
|| this._steeringPriorEffectObserved(tabId, prior);
}
@@ -5829,18 +5834,20 @@ export class Agent extends LoopDetector {
&& !this._isScheduledResumeTurn(message)));
}
- // The platforms of every attempt the current task has dispatched, from its
- // first run on, or platforms null when WebBrain cannot know them all. In a
- // session the record follows the task through its runs, corrections and
- // Continue. Without one, for example after WebBrain restarted, only a task
- // that has not run since its request, or since the correction whose receipt
- // lists the platforms before it, is known.
+ // What the current task has attempted, from its first run on: the sites of
+ // every dispatched attempt that may have changed something (platforms) and
+ // of every publication among them (publications). Either is null when
+ // WebBrain cannot know it. The record follows the task through its runs,
+ // corrections and Continue, and is saved with the conversation. Without a
+ // record, as for a conversation saved by an earlier version, only a task
+ // that has not run since its request is known, and of a correction with
+ // nothing run since it only the sites its receipt lists.
_taskAttemptLedger(tabId, messages = this.conversations?.get(tabId) || [], binding = this._activeTaskBinding(messages)) {
const key = this._taskAttemptLedgerKey(tabId, messages, binding);
const ledgers = this._taskAttemptLedgers ||= new Map();
const entry = ledgers.get(tabId);
if (key && entry?.key === key) return entry;
- let platforms = null;
+ let platforms = null, publications = null;
if (key) {
const prior = binding.priorSubmission;
let from = binding.taskIndex;
@@ -5851,46 +5858,96 @@ export class Agent extends LoopDetector {
}
}
if (this._noTaskRunSince(messages, from)) {
- if (!prior) platforms = [];
+ if (!prior) { platforms = []; publications = []; }
else if (Array.isArray(prior.taskAttemptPlatforms)) platforms = [...prior.taskAttemptPlatforms];
}
}
- const created = { key, platforms };
+ const created = { key, platforms, publications };
ledgers.set(tabId, created);
return created;
}
// Adds what an attempt may have changed to the task's record: a dispatched
// submission, publication or deletion that commits something, a fetch that
- // sends a mutation, and a store publication. A search commits nothing.
+ // sends a mutation, and a store publication. A search, a send the browser
+ // refused and a form the page rejected commit nothing.
_recordTaskAttempt(tabId, name, args, detected, result, submitState) {
try {
- this._addTaskAttempt(tabId, name, args, detected, result, submitState);
+ const keys = new Set();
+ let publication = false;
+ if (submitState?.dispatched === true && submitState.formValidationFailed !== true && submitState.nonCommitting !== true) {
+ for (const key of this._attemptPlatformKeys(submitState.publicationSnapshot?.account,
+ [submitState.originatingUrl, detected?.url, detected?.frameUrl, submitState.iframeSubmission?.currentUrl])) keys.add(key);
+ publication = detected?.publicationControl === true || submitState.mutationKind === 'publish';
+ }
+ if (isNetworkMutation(name, args) || name === 'chrome_web_store_publish') {
+ const notSent = result?.noDispatch === true || result?.denied || result?.skipped || result?.cancelled
+ || (result?.dispatched === false && result?.outcomeUnknown !== true && result?.missingToolResponse !== true);
+ // A request goes to its own address, not the page's, and may publish there.
+ if (!notSent) {
+ keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'
+ ? 'https://chromewebstore.google.com/' : args?.url) || '*');
+ publication = true;
+ }
+ }
+ if (keys.size) this._addTaskAttemptKeys(tabId, [...keys], publication);
} catch {
- // A record that could not be updated is no longer complete; the action
- // itself is unaffected.
- try { const entry = this._taskAttemptLedgers?.get(tabId); if (entry) entry.platforms = null; } catch {}
+ this._markTaskAttemptsUnknown(tabId);
}
}
- _addTaskAttempt(tabId, name, args, detected, result, submitState) {
- const keys = new Set();
- if (submitState?.dispatched === true && !this._nonCommittingSubmission(detected)) {
- for (const key of this._attemptPlatformKeys(submitState.publicationSnapshot?.account,
- [submitState.originatingUrl, detected?.url, detected?.frameUrl, submitState.iframeSubmission?.currentUrl])) keys.add(key);
- }
- if (isNetworkMutation(name, args) || name === 'chrome_web_store_publish') {
- const notSent = result?.noDispatch === true || result?.denied || result?.skipped || result?.cancelled
- || (result?.dispatched === false && result?.outcomeUnknown !== true && result?.missingToolResponse !== true);
- // A request goes to its own address, not the page's.
- if (!notSent) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'
+ // An action that may have been sent but whose result was never recorded,
+ // because Stop or an error interrupted it: its site joins the record when
+ // it may have committed something, and an unknown site makes it unknown.
+ _recordInterruptedTaskAttempt(tabId, name, args, detected) {
+ try {
+ const network = isNetworkMutation(name, args) || name === 'chrome_web_store_publish';
+ const commit = (detected?.isSubmit === true && !this._nonCommittingSubmission(detected))
+ || name === 'execute_js' || this._formValidationActionLooksSubmit(name, args, null, detected);
+ if (!commit && !network) return;
+ const keys = new Set(commit ? this._attemptPlatformKeys(detected?.publicationSnapshot?.account, [detected?.url, detected?.frameUrl]) : []);
+ if (network) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'
? 'https://chromewebstore.google.com/' : args?.url) || '*');
+ this._addTaskAttemptKeys(tabId, [...keys], network || detected?.publicationControl === true || name === 'execute_js');
+ } catch {
+ this._markTaskAttemptsUnknown(tabId);
}
- if (!keys.size) return;
+ }
+
+ _addTaskAttemptKeys(tabId, keys, publication) {
const ledger = this._taskAttemptLedger(tabId);
- if (!Array.isArray(ledger.platforms)) return;
- const platforms = [...new Set([...ledger.platforms, ...keys])];
- ledger.platforms = platforms.length > 64 ? null : platforms;
+ const add = (list, values) => {
+ if (!Array.isArray(list)) return null;
+ const merged = [...new Set([...list, ...values])];
+ return merged.length > 64 ? null : merged;
+ };
+ ledger.platforms = add(ledger.platforms, keys);
+ if (publication) ledger.publications = add(ledger.publications, keys);
+ else if (keys.includes('*')) ledger.publications = add(ledger.publications, ['*']);
+ }
+
+ // A record that could not be updated is no longer complete; the action
+ // itself is unaffected.
+ _markTaskAttemptsUnknown(tabId) {
+ try {
+ const entry = this._taskAttemptLedgers?.get(tabId);
+ if (entry) { entry.platforms = null; entry.publications = null; }
+ } catch {}
+ }
+
+ // The record as saved with the conversation, and as read back.
+ _persistedTaskAttemptLedger(tabId) {
+ const entry = this._taskAttemptLedgers?.get(tabId);
+ return entry?.key ? { key: entry.key,
+ platforms: Array.isArray(entry.platforms) ? [...entry.platforms] : null,
+ publications: Array.isArray(entry.publications) ? [...entry.publications] : null } : null;
+ }
+
+ _restoreTaskAttemptLedger(tabId, saved) {
+ const list = value => Array.isArray(value) && value.length <= 64 && value.every(item => typeof item === 'string' && item.length <= 300)
+ ? [...value] : null;
+ if (!saved || typeof saved !== 'object' || typeof saved.key !== 'string' || !saved.key || saved.key.length > 300) return;
+ (this._taskAttemptLedgers ||= new Map()).set(tabId, { key: saved.key, platforms: list(saved.platforms), publications: list(saved.publications) });
}
// The platforms of every attempt the task made up to a correction, its
@@ -5899,7 +5956,8 @@ export class Agent extends LoopDetector {
try {
const ledger = this._taskAttemptLedger(tabId, messages, binding).platforms;
if (!Array.isArray(ledger)) return null;
- const platforms = [...new Set([...ledger, ...(receipt ? this._receiptPlatformKeys(receipt) : [])])];
+ const committed = receipt && receipt.nonCommitting !== true && receipt.formValidationFailed !== true;
+ const platforms = [...new Set([...ledger, ...(committed ? this._receiptPlatformKeys(receipt) : [])])];
return platforms.length > 64 ? null : platforms;
} catch {
return null;
@@ -5965,13 +6023,21 @@ export class Agent extends LoopDetector {
const outcomePlatforms = archive => (archive?.contract?.actions || [])
.filter(action => action?.id && archive.outcomes?.[action.id] && archive.outcomes[action.id].status !== 'failed')
.map(action => action.platform || '*');
- const earlierPlatforms = [...(attempted || []),
+ // Posts made under the correction itself: in the run that applied it, or
+ // a Continue that carries that run, the publication contract's outcomes
+ // and the dispatch records stop a repeat. A run rebuilt from the
+ // conversation has neither, so the task's record of publications does.
+ const sinceCorrection = guard.steeringHistoryInMemory === true ? []
+ : this._taskAttemptLedger(tabId).publications;
+ const earlierPlatforms = [...(attempted || []), ...(sinceCorrection || []),
...[guard.steeringPriorPublication, guard.steeringPriorContext?.socialPublication].flatMap(outcomePlatforms),
- ...(guard.steeringMutationReceipts || []).filter(receipt => receipt?.dispatched === true)
+ ...(guard.steeringMutationReceipts || []).filter(receipt => receipt?.dispatched === true
+ && receipt.nonCommitting !== true && receipt.formValidationFailed !== true)
.flatMap(receipt => this._receiptPlatformKeys(receipt))];
const earlierAttemptHere = earlierPlatforms.some(platform => currentPlatforms.includes(platform));
- const historyKnown = !!attempted && !earlierPlatforms.includes('*');
- const crossPlatform = accountCrossPlatform || (otherSite && historyKnown && !earlierAttemptHere);
+ const historyKnown = !!attempted && Array.isArray(sinceCorrection) && !earlierPlatforms.includes('*');
+ const siteCrossPlatform = otherSite && historyKnown && !earlierAttemptHere;
+ const crossPlatform = accountCrossPlatform || siteCrossPlatform;
if (publication && priorSnapshot?.account && detected.publicationSnapshot?.account !== priorSnapshot.account && !crossPlatform && !otherSite) {
const account = String(detected.publicationSnapshot?.account || '').split(':').slice(1).join(':');
if (!account || !latest.toLowerCase().includes(account.toLowerCase())) return blocked('replacement_account_changed');
@@ -6037,8 +6103,8 @@ export class Agent extends LoopDetector {
// to look for the other site's post on this page.
if (prior && publication && otherSite && !crossPlatform) {
return blocked(earlierAttemptHere ? 'earlier_attempt_on_platform' : 'attempt_history_unknown', earlierAttemptHere
- ? `Nothing was published. Before the latest correction this task already made an attempt on this site, and only the latest attempt${priorHost ? `, on ${priorHost},` : ''} can be checked, so this post could repeat the earlier one or sit beside it. Do not publish it again. Tell the user what was not done and that the earlier attempt on this site may already be there; they can check it and ask again.`
- : `Nothing was published. WebBrain has no full record of what this task attempted before the latest correction (for example after WebBrain restarted), so it cannot rule out an earlier attempt on this site, and the latest attempt${priorHost ? `, on ${priorHost},` : ''} cannot be checked here. Do not publish it again. Tell the user what was not done; they can ask again to post it.`,
+ ? `Nothing was published. This task already made an attempt on this site, and WebBrain can check only the latest attempt from before the correction${priorHost ? `, on ${priorHost}` : ''}, so this post could repeat the earlier one or sit beside it. Do not publish it again. Tell the user what was not done and that the earlier attempt on this site may already be there; they can check it and ask again.`
+ : `Nothing was published. WebBrain has no full record of what this task attempted (for example after WebBrain restarted), so it cannot rule out an earlier attempt on this site, and the latest attempt from before the correction${priorHost ? `, on ${priorHost},` : ''} cannot be checked here. Do not publish it again. Tell the user what was not done; they can ask again to post it.`,
{ retryable: false });
}
if (prior) {
@@ -6066,16 +6132,19 @@ export class Agent extends LoopDetector {
: genericCommit && (!!priorSnapshot || !priorForm || priorForm !== detected.actionUrl);
const live = submitted && submitted.actionSequence === prior.actionSequence ? submitted : prior;
const inPlace = !!live.currentUrl && this._normalizeUrl(live.currentUrl) === this._normalizeUrl(live.originatingUrl || '');
- if (differentTarget && this._steeringPriorEffectKnown(tabId, guard, prior)) observedKind = publication ? 'cross_platform' : 'distinct_target';
- else if (formCommit && priorForm && priorForm === detected.actionUrl && inPlace
+ // A page read under the correction counts only where the record
+ // shows the post cannot repeat an earlier attempt.
+ if (differentTarget && this._steeringPriorEffectKnown(tabId, guard, prior, { readAfterCorrection: publication && siteCrossPlatform })) {
+ observedKind = publication ? 'cross_platform' : 'distinct_target';
+ } else if (formCommit && priorForm && priorForm === detected.actionUrl && inPlace
&& this._steeringPriorEffectObserved(tabId, prior)) observedKind = 'form_observation';
}
if (prior.dispatched !== true || (matching.length !== 1 && !deletedVerifiedResource && !observedKind)) {
return blocked('prior_effect_unreconciled', publication && !crossPlatform
? 'The earlier post from before the latest correction has not been matched yet. Read it on this platform (its permalink or the account\'s recent posts), then retry; do not repeat an uncertain publication.'
- : crossPlatform
- // Any page read since the correction is enough for a post on
- // another platform, so the model is not sent back to the other site.
+ : publication && siteCrossPlatform && prior.mutationKind !== 'delete'
+ // Any page read since the correction is enough here, so the model
+ // is not sent back to the other site.
? `The earlier submission from before the latest correction${priorHost ? `, on ${priorHost},` : ''} has not been observed yet: no page was read since it was sent. Read the current page, then retry; nothing about it needs to be found here, and do not repeat it.`
: 'The earlier submission from before the latest correction has not been observed yet. Read the page that shows its result, then retry; do not repeat an uncertain mutation.');
}
@@ -13156,6 +13225,15 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this._throwIfAborted(this._runAbortSignal(tabId));
return result;
} catch (error) {
+ // An action that was sent, or may have been, before Stop or an error
+ // cut its bookkeeping short still joins the task's record of attempts.
+ const interrupted = cancellationState.active;
+ if (interrupted?.invoked && !interrupted.attemptRecorded) {
+ const raw = interrupted.result || interrupted.dispatchState?.rawToolResult;
+ const unsent = raw?.noDispatch === true || raw?.dispatched === false
+ || (!raw && interrupted.dispatchState?.tracksTextMutationDispatch === true && interrupted.dispatchState.started !== true);
+ if (!unsent) this._recordInterruptedTaskAttempt(tabId, interrupted.name, interrupted.args, interrupted.detected);
+ }
if (this._checkAbort(tabId) || error?.name === 'AbortError') {
// Repair the whole assistant tool batch before any caller persists a
// terminal cancellation. Earlier completed results must stay intact.
@@ -14518,6 +14596,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
callState.consequential = missingResponseOutcomeUnknown;
callState.args = fnArgs;
+ callState.name = fnName;
+ callState.detected = detectedSubmitAction;
onUpdate('tool_call', {
name: fnName,
args: fnArgs,
@@ -14879,6 +14959,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
beforeDocument, String(this._lastAxScopes.get(tabId)?.documentToken || ''),
messageRecipientExecutionContext,
);
+ callState.attemptRecorded = true;
if (toolResult && typeof toolResult === 'object' && !toolResult.done) {
bulkApiShortcut = this._detectBulkApiMutationShortcut(tabId, fnName, fnArgs, toolResult, {
startedAt: _toolStart,
@@ -18704,6 +18785,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
});
}
this._restorePersistedRichTextToolbarAudit(tabId, entry.richTextToolbarAudit);
+ this._restoreTaskAttemptLedger(tabId, entry.taskAttemptLedger);
const savedDispatch = entry.nativeCaptchaDispatch;
const rootDocument = Array.isArray(savedDispatch?.documents)
? savedDispatch.documents.find(document => document?.frameId === 0) : null;
@@ -18841,6 +18923,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
clarificationAuthorizationGuard: persistedClarificationGuard,
continuationResponseLanguagePolicy: persistedContinuationLanguage,
richTextToolbarAudit: this._persistedRichTextToolbarAudit(tabId),
+ // The sites the current task attempted, so a correction after a restart
+ // still knows them; never authorization.
+ taskAttemptLedger: this._persistedTaskAttemptLedger(tabId),
captchaGateState: captchaGateState?.cloudflareManagedChallenge === true
|| captchaGateState?.publicGate?.cloudflareManagedChallenge === true
? null
@@ -34843,6 +34928,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// a read in this run.
steeringPriorReadAfterCorrection: carryMatches && !!carried.steeringPriorSubmission
&& carried.steeringPriorReadAfterCorrection === true,
+ // Only a Continue that carries the run's contract and dispatch records
+ // keeps them; any other start rebuilds the guard from the conversation.
+ steeringHistoryInMemory: carryMatches && !!carried.steeringPriorSubmission
+ && carried.steeringHistoryInMemory === true,
steeringMutationReceipts: carryMatches ? structuredClone(carried.steeringMutationReceipts || []) : [],
steeringPriorContext: carryMatches ? structuredClone(carried.steeringPriorContext || null) : null,
steeringPriorSubmissionReconciliation: carryMatches ? structuredClone(carried.steeringPriorSubmissionReconciliation || null) : null,
@@ -35376,6 +35465,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
verifiedSubmissionEvidence: guard.verifiedSubmissionEvidence === true,
steeringPriorSubmission: guard.steeringPriorSubmission ? structuredClone(guard.steeringPriorSubmission) : null,
steeringPriorReadAfterCorrection: guard.steeringPriorReadAfterCorrection === true,
+ steeringHistoryInMemory: guard.steeringHistoryInMemory === true,
steeringMutationReceipts: structuredClone(guard.steeringMutationReceipts || []),
steeringPriorContext: structuredClone(guard.steeringPriorContext || null),
steeringPriorSubmissionReconciliation: structuredClone(guard.steeringPriorSubmissionReconciliation || null),
@@ -40145,6 +40235,19 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (blocked) return blocked;
}
+ // The page's leave prompt is answered only through the run's debugger
+ // connection: arm it again when the run started where none was
+ // possible, such as chrome://extensions, or the connection dropped.
+ // This comes before anything listens for the navigation, so a page
+ // that navigates by itself meanwhile is not taken for this one.
+ await cdpClient.ensureDialogHandling(tabId);
+ // Stop while dialog handling was being armed ends the run here,
+ // before anything is sent.
+ if (this._checkAbort(tabId) || earlyCdpAbortSignal?.aborted) {
+ return { success: false, dispatched: false, noDispatch: true, cancelled: true,
+ error: 'navigate: stopped before the navigation was sent.' };
+ }
+
let beforeUrl = '';
try {
const tab = await chrome.tabs.get(tabId);
@@ -40245,17 +40348,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
};
try {
- // The page's leave prompt is answered only through the run's debugger
- // connection: arm it again when the run started where none was
- // possible, such as chrome://extensions, or the connection dropped.
- await cdpClient.ensureDialogHandling(tabId);
- // Stop while dialog handling was being armed ends the run here,
- // before anything is sent.
- if (this._checkAbort(tabId) || earlyCdpAbortSignal?.aborted) {
- removeNavigationListener();
- return { success: false, dispatched: false, noDispatch: true, cancelled: true,
- error: 'navigate: stopped before the navigation was sent.' };
- }
releaseDialogNavigation = cdpClient.authorizeNavigationDialog(tabId, beforeUrl, earlyCdpAbortSignal);
await beforePageAgentDispatch(globalThis.chrome, tabId, { kind: 'navigate', url: rawUrl });
await chrome.tabs.update(tabId, { url: rawUrl });
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 38cdf3fcb..cc416339a 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -1809,6 +1809,8 @@ export class Agent extends LoopDetector {
actionRef: name !== 'iframe_click' && /^ref_\d+$/.test(String(args?.ref_id || '')) ? args.ref_id : '',
actionSequence: Number(this.completionInvariants.get(tabId)?.lastAction?.sequence || 0),
submitLike: true,
+ // A search or an in-form toggle reaches here too; it commits nothing.
+ ...(this._nonCommittingSubmission(detectedSubmit) ? { nonCommitting: true } : {}),
semanticCommit: semanticCommit ? {
taskKey: this._progressTaskKeyHash(tabId), runToken: this.completionInvariants.get(tabId)?.runToken,
sourceSha256: this._sha256TextSync(this._progressTaskAnchorText(tabId)),
@@ -5180,6 +5182,9 @@ export class Agent extends LoopDetector {
guard.steeringPriorSubmission = priorSubmission ? { ...structuredClone(priorSubmission),
taskAttemptPlatforms: this._correctionAttemptPlatforms(tabId, messages, this._activeTaskBinding(messages), priorSubmission) } : null;
guard.steeringPriorReadAfterCorrection = false;
+ // This run applied the correction: what it sends under it stays in its
+ // publication contract and dispatch records.
+ guard.steeringHistoryInMemory = true;
guard.steeringMutationReceipts = structuredClone(previous?.steeringMutationReceipts || []);
// Action sequences restart with each run, so a receipt is one attempt
// only within its run.
@@ -5304,13 +5309,13 @@ export class Agent extends LoopDetector {
// Outcome knowledge that does not depend on the current page: a cached
// reconciliation of this exact receipt, a job-verified publication outcome,
- // or an explicit page observation after the dispatch, including any page
- // read under the correction, which came after it.
- _steeringPriorEffectKnown(tabId, guard, prior) {
+ // or an explicit page observation after the dispatch. Where the caller asks,
+ // any page read under the correction counts too, since it came after it.
+ _steeringPriorEffectKnown(tabId, guard, prior, { readAfterCorrection = false } = {}) {
const cached = guard?.steeringPriorSubmissionReconciliation;
return (cached?.verified === true && cached.actionSequence === prior?.actionSequence)
|| this._steeringPriorPublicationOutcome(guard, prior) === 'verified'
- || (!!prior && guard?.steeringPriorSubmission === prior && guard.steeringPriorReadAfterCorrection === true)
+ || (readAfterCorrection && !!prior && guard?.steeringPriorSubmission === prior && guard.steeringPriorReadAfterCorrection === true)
|| this._steeringPriorEffectObserved(tabId, prior);
}
@@ -5502,18 +5507,20 @@ export class Agent extends LoopDetector {
&& !this._isScheduledResumeTurn(message)));
}
- // The platforms of every attempt the current task has dispatched, from its
- // first run on, or platforms null when WebBrain cannot know them all. In a
- // session the record follows the task through its runs, corrections and
- // Continue. Without one, for example after WebBrain restarted, only a task
- // that has not run since its request, or since the correction whose receipt
- // lists the platforms before it, is known.
+ // What the current task has attempted, from its first run on: the sites of
+ // every dispatched attempt that may have changed something (platforms) and
+ // of every publication among them (publications). Either is null when
+ // WebBrain cannot know it. The record follows the task through its runs,
+ // corrections and Continue, and is saved with the conversation. Without a
+ // record, as for a conversation saved by an earlier version, only a task
+ // that has not run since its request is known, and of a correction with
+ // nothing run since it only the sites its receipt lists.
_taskAttemptLedger(tabId, messages = this.conversations?.get(tabId) || [], binding = this._activeTaskBinding(messages)) {
const key = this._taskAttemptLedgerKey(tabId, messages, binding);
const ledgers = this._taskAttemptLedgers ||= new Map();
const entry = ledgers.get(tabId);
if (key && entry?.key === key) return entry;
- let platforms = null;
+ let platforms = null, publications = null;
if (key) {
const prior = binding.priorSubmission;
let from = binding.taskIndex;
@@ -5524,46 +5531,96 @@ export class Agent extends LoopDetector {
}
}
if (this._noTaskRunSince(messages, from)) {
- if (!prior) platforms = [];
+ if (!prior) { platforms = []; publications = []; }
else if (Array.isArray(prior.taskAttemptPlatforms)) platforms = [...prior.taskAttemptPlatforms];
}
}
- const created = { key, platforms };
+ const created = { key, platforms, publications };
ledgers.set(tabId, created);
return created;
}
// Adds what an attempt may have changed to the task's record: a dispatched
// submission, publication or deletion that commits something, a fetch that
- // sends a mutation, and a store publication. A search commits nothing.
+ // sends a mutation, and a store publication. A search, a send the browser
+ // refused and a form the page rejected commit nothing.
_recordTaskAttempt(tabId, name, args, detected, result, submitState) {
try {
- this._addTaskAttempt(tabId, name, args, detected, result, submitState);
+ const keys = new Set();
+ let publication = false;
+ if (submitState?.dispatched === true && submitState.formValidationFailed !== true && submitState.nonCommitting !== true) {
+ for (const key of this._attemptPlatformKeys(submitState.publicationSnapshot?.account,
+ [submitState.originatingUrl, detected?.url, detected?.frameUrl, submitState.iframeSubmission?.currentUrl])) keys.add(key);
+ publication = detected?.publicationControl === true || submitState.mutationKind === 'publish';
+ }
+ if (isNetworkMutation(name, args) || name === 'chrome_web_store_publish') {
+ const notSent = result?.noDispatch === true || result?.denied || result?.skipped || result?.cancelled
+ || (result?.dispatched === false && result?.outcomeUnknown !== true && result?.missingToolResponse !== true);
+ // A request goes to its own address, not the page's, and may publish there.
+ if (!notSent) {
+ keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'
+ ? 'https://chromewebstore.google.com/' : args?.url) || '*');
+ publication = true;
+ }
+ }
+ if (keys.size) this._addTaskAttemptKeys(tabId, [...keys], publication);
} catch {
- // A record that could not be updated is no longer complete; the action
- // itself is unaffected.
- try { const entry = this._taskAttemptLedgers?.get(tabId); if (entry) entry.platforms = null; } catch {}
+ this._markTaskAttemptsUnknown(tabId);
}
}
- _addTaskAttempt(tabId, name, args, detected, result, submitState) {
- const keys = new Set();
- if (submitState?.dispatched === true && !this._nonCommittingSubmission(detected)) {
- for (const key of this._attemptPlatformKeys(submitState.publicationSnapshot?.account,
- [submitState.originatingUrl, detected?.url, detected?.frameUrl, submitState.iframeSubmission?.currentUrl])) keys.add(key);
- }
- if (isNetworkMutation(name, args) || name === 'chrome_web_store_publish') {
- const notSent = result?.noDispatch === true || result?.denied || result?.skipped || result?.cancelled
- || (result?.dispatched === false && result?.outcomeUnknown !== true && result?.missingToolResponse !== true);
- // A request goes to its own address, not the page's.
- if (!notSent) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'
+ // An action that may have been sent but whose result was never recorded,
+ // because Stop or an error interrupted it: its site joins the record when
+ // it may have committed something, and an unknown site makes it unknown.
+ _recordInterruptedTaskAttempt(tabId, name, args, detected) {
+ try {
+ const network = isNetworkMutation(name, args) || name === 'chrome_web_store_publish';
+ const commit = (detected?.isSubmit === true && !this._nonCommittingSubmission(detected))
+ || name === 'execute_js' || this._formValidationActionLooksSubmit(name, args, null, detected);
+ if (!commit && !network) return;
+ const keys = new Set(commit ? this._attemptPlatformKeys(detected?.publicationSnapshot?.account, [detected?.url, detected?.frameUrl]) : []);
+ if (network) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'
? 'https://chromewebstore.google.com/' : args?.url) || '*');
+ this._addTaskAttemptKeys(tabId, [...keys], network || detected?.publicationControl === true || name === 'execute_js');
+ } catch {
+ this._markTaskAttemptsUnknown(tabId);
}
- if (!keys.size) return;
+ }
+
+ _addTaskAttemptKeys(tabId, keys, publication) {
const ledger = this._taskAttemptLedger(tabId);
- if (!Array.isArray(ledger.platforms)) return;
- const platforms = [...new Set([...ledger.platforms, ...keys])];
- ledger.platforms = platforms.length > 64 ? null : platforms;
+ const add = (list, values) => {
+ if (!Array.isArray(list)) return null;
+ const merged = [...new Set([...list, ...values])];
+ return merged.length > 64 ? null : merged;
+ };
+ ledger.platforms = add(ledger.platforms, keys);
+ if (publication) ledger.publications = add(ledger.publications, keys);
+ else if (keys.includes('*')) ledger.publications = add(ledger.publications, ['*']);
+ }
+
+ // A record that could not be updated is no longer complete; the action
+ // itself is unaffected.
+ _markTaskAttemptsUnknown(tabId) {
+ try {
+ const entry = this._taskAttemptLedgers?.get(tabId);
+ if (entry) { entry.platforms = null; entry.publications = null; }
+ } catch {}
+ }
+
+ // The record as saved with the conversation, and as read back.
+ _persistedTaskAttemptLedger(tabId) {
+ const entry = this._taskAttemptLedgers?.get(tabId);
+ return entry?.key ? { key: entry.key,
+ platforms: Array.isArray(entry.platforms) ? [...entry.platforms] : null,
+ publications: Array.isArray(entry.publications) ? [...entry.publications] : null } : null;
+ }
+
+ _restoreTaskAttemptLedger(tabId, saved) {
+ const list = value => Array.isArray(value) && value.length <= 64 && value.every(item => typeof item === 'string' && item.length <= 300)
+ ? [...value] : null;
+ if (!saved || typeof saved !== 'object' || typeof saved.key !== 'string' || !saved.key || saved.key.length > 300) return;
+ (this._taskAttemptLedgers ||= new Map()).set(tabId, { key: saved.key, platforms: list(saved.platforms), publications: list(saved.publications) });
}
// The platforms of every attempt the task made up to a correction, its
@@ -5572,7 +5629,8 @@ export class Agent extends LoopDetector {
try {
const ledger = this._taskAttemptLedger(tabId, messages, binding).platforms;
if (!Array.isArray(ledger)) return null;
- const platforms = [...new Set([...ledger, ...(receipt ? this._receiptPlatformKeys(receipt) : [])])];
+ const committed = receipt && receipt.nonCommitting !== true && receipt.formValidationFailed !== true;
+ const platforms = [...new Set([...ledger, ...(committed ? this._receiptPlatformKeys(receipt) : [])])];
return platforms.length > 64 ? null : platforms;
} catch {
return null;
@@ -5638,13 +5696,21 @@ export class Agent extends LoopDetector {
const outcomePlatforms = archive => (archive?.contract?.actions || [])
.filter(action => action?.id && archive.outcomes?.[action.id] && archive.outcomes[action.id].status !== 'failed')
.map(action => action.platform || '*');
- const earlierPlatforms = [...(attempted || []),
+ // Posts made under the correction itself: in the run that applied it, or
+ // a Continue that carries that run, the publication contract's outcomes
+ // and the dispatch records stop a repeat. A run rebuilt from the
+ // conversation has neither, so the task's record of publications does.
+ const sinceCorrection = guard.steeringHistoryInMemory === true ? []
+ : this._taskAttemptLedger(tabId).publications;
+ const earlierPlatforms = [...(attempted || []), ...(sinceCorrection || []),
...[guard.steeringPriorPublication, guard.steeringPriorContext?.socialPublication].flatMap(outcomePlatforms),
- ...(guard.steeringMutationReceipts || []).filter(receipt => receipt?.dispatched === true)
+ ...(guard.steeringMutationReceipts || []).filter(receipt => receipt?.dispatched === true
+ && receipt.nonCommitting !== true && receipt.formValidationFailed !== true)
.flatMap(receipt => this._receiptPlatformKeys(receipt))];
const earlierAttemptHere = earlierPlatforms.some(platform => currentPlatforms.includes(platform));
- const historyKnown = !!attempted && !earlierPlatforms.includes('*');
- const crossPlatform = accountCrossPlatform || (otherSite && historyKnown && !earlierAttemptHere);
+ const historyKnown = !!attempted && Array.isArray(sinceCorrection) && !earlierPlatforms.includes('*');
+ const siteCrossPlatform = otherSite && historyKnown && !earlierAttemptHere;
+ const crossPlatform = accountCrossPlatform || siteCrossPlatform;
if (publication && priorSnapshot?.account && detected.publicationSnapshot?.account !== priorSnapshot.account && !crossPlatform && !otherSite) {
const account = String(detected.publicationSnapshot?.account || '').split(':').slice(1).join(':');
if (!account || !latest.toLowerCase().includes(account.toLowerCase())) return blocked('replacement_account_changed');
@@ -5710,8 +5776,8 @@ export class Agent extends LoopDetector {
// to look for the other site's post on this page.
if (prior && publication && otherSite && !crossPlatform) {
return blocked(earlierAttemptHere ? 'earlier_attempt_on_platform' : 'attempt_history_unknown', earlierAttemptHere
- ? `Nothing was published. Before the latest correction this task already made an attempt on this site, and only the latest attempt${priorHost ? `, on ${priorHost},` : ''} can be checked, so this post could repeat the earlier one or sit beside it. Do not publish it again. Tell the user what was not done and that the earlier attempt on this site may already be there; they can check it and ask again.`
- : `Nothing was published. WebBrain has no full record of what this task attempted before the latest correction (for example after WebBrain restarted), so it cannot rule out an earlier attempt on this site, and the latest attempt${priorHost ? `, on ${priorHost},` : ''} cannot be checked here. Do not publish it again. Tell the user what was not done; they can ask again to post it.`,
+ ? `Nothing was published. This task already made an attempt on this site, and WebBrain can check only the latest attempt from before the correction${priorHost ? `, on ${priorHost}` : ''}, so this post could repeat the earlier one or sit beside it. Do not publish it again. Tell the user what was not done and that the earlier attempt on this site may already be there; they can check it and ask again.`
+ : `Nothing was published. WebBrain has no full record of what this task attempted (for example after WebBrain restarted), so it cannot rule out an earlier attempt on this site, and the latest attempt from before the correction${priorHost ? `, on ${priorHost},` : ''} cannot be checked here. Do not publish it again. Tell the user what was not done; they can ask again to post it.`,
{ retryable: false });
}
if (prior) {
@@ -5739,16 +5805,19 @@ export class Agent extends LoopDetector {
: genericCommit && (!!priorSnapshot || !priorForm || priorForm !== detected.actionUrl);
const live = submitted && submitted.actionSequence === prior.actionSequence ? submitted : prior;
const inPlace = !!live.currentUrl && this._normalizeUrl(live.currentUrl) === this._normalizeUrl(live.originatingUrl || '');
- if (differentTarget && this._steeringPriorEffectKnown(tabId, guard, prior)) observedKind = publication ? 'cross_platform' : 'distinct_target';
- else if (formCommit && priorForm && priorForm === detected.actionUrl && inPlace
+ // A page read under the correction counts only where the record
+ // shows the post cannot repeat an earlier attempt.
+ if (differentTarget && this._steeringPriorEffectKnown(tabId, guard, prior, { readAfterCorrection: publication && siteCrossPlatform })) {
+ observedKind = publication ? 'cross_platform' : 'distinct_target';
+ } else if (formCommit && priorForm && priorForm === detected.actionUrl && inPlace
&& this._steeringPriorEffectObserved(tabId, prior)) observedKind = 'form_observation';
}
if (prior.dispatched !== true || (matching.length !== 1 && !deletedVerifiedResource && !observedKind)) {
return blocked('prior_effect_unreconciled', publication && !crossPlatform
? 'The earlier post from before the latest correction has not been matched yet. Read it on this platform (its permalink or the account\'s recent posts), then retry; do not repeat an uncertain publication.'
- : crossPlatform
- // Any page read since the correction is enough for a post on
- // another platform, so the model is not sent back to the other site.
+ : publication && siteCrossPlatform && prior.mutationKind !== 'delete'
+ // Any page read since the correction is enough here, so the model
+ // is not sent back to the other site.
? `The earlier submission from before the latest correction${priorHost ? `, on ${priorHost},` : ''} has not been observed yet: no page was read since it was sent. Read the current page, then retry; nothing about it needs to be found here, and do not repeat it.`
: 'The earlier submission from before the latest correction has not been observed yet. Read the page that shows its result, then retry; do not repeat an uncertain mutation.');
}
@@ -6370,6 +6439,7 @@ export class Agent extends LoopDetector {
});
}
this._restorePersistedRichTextToolbarAudit(tabId, entry.richTextToolbarAudit);
+ this._restoreTaskAttemptLedger(tabId, entry.taskAttemptLedger);
const savedDispatch = entry.nativeCaptchaDispatch;
const rootDocument = Array.isArray(savedDispatch?.documents)
? savedDispatch.documents.find(document => document?.frameId === 0) : null;
@@ -6507,6 +6577,9 @@ export class Agent extends LoopDetector {
clarificationAuthorizationGuard: persistedClarificationGuard,
continuationResponseLanguagePolicy: persistedContinuationLanguage,
richTextToolbarAudit: this._persistedRichTextToolbarAudit(tabId),
+ // The sites the current task attempted, so a correction after a restart
+ // still knows them; never authorization.
+ taskAttemptLedger: this._persistedTaskAttemptLedger(tabId),
captchaGateState: captchaGateState?.cloudflareManagedChallenge === true
|| captchaGateState?.publicGate?.cloudflareManagedChallenge === true
? null
@@ -11794,6 +11867,15 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this._throwIfAborted(this._runAbortSignal(tabId));
return result;
} catch (error) {
+ // An action that was sent, or may have been, before Stop or an error
+ // cut its bookkeeping short still joins the task's record of attempts.
+ const interrupted = cancellationState.active;
+ if (interrupted?.invoked && !interrupted.attemptRecorded) {
+ const raw = interrupted.result || interrupted.dispatchState?.rawToolResult;
+ const unsent = raw?.noDispatch === true || raw?.dispatched === false
+ || (!raw && interrupted.dispatchState?.tracksTextMutationDispatch === true && interrupted.dispatchState.started !== true);
+ if (!unsent) this._recordInterruptedTaskAttempt(tabId, interrupted.name, interrupted.args, interrupted.detected);
+ }
if (this._checkAbort(tabId) || error?.name === 'AbortError') {
// Repair the whole assistant tool batch before any caller persists a
// terminal cancellation. Earlier completed results must stay intact.
@@ -13081,6 +13163,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
callState.consequential = missingResponseOutcomeUnknown;
callState.args = fnArgs;
+ callState.name = fnName;
+ callState.detected = detectedSubmitAction;
onUpdate('tool_call', {
name: fnName,
args: fnArgs,
@@ -13422,6 +13506,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
beforeDocument, String(this._lastAxScopes.get(tabId)?.documentToken || ''),
messageRecipientExecutionContext,
);
+ callState.attemptRecorded = true;
if (toolResult && typeof toolResult === 'object' && !toolResult.done) {
bulkApiShortcut = this._detectBulkApiMutationShortcut(tabId, fnName, fnArgs, toolResult, {
startedAt: _toolStart,
@@ -32122,6 +32207,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// a read in this run.
steeringPriorReadAfterCorrection: carryMatches && !!carried.steeringPriorSubmission
&& carried.steeringPriorReadAfterCorrection === true,
+ // Only a Continue that carries the run's contract and dispatch records
+ // keeps them; any other start rebuilds the guard from the conversation.
+ steeringHistoryInMemory: carryMatches && !!carried.steeringPriorSubmission
+ && carried.steeringHistoryInMemory === true,
steeringMutationReceipts: carryMatches ? structuredClone(carried.steeringMutationReceipts || []) : [],
steeringPriorContext: carryMatches ? structuredClone(carried.steeringPriorContext || null) : null,
steeringPriorSubmissionReconciliation: carryMatches ? structuredClone(carried.steeringPriorSubmissionReconciliation || null) : null,
@@ -32655,6 +32744,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
verifiedSubmissionEvidence: guard.verifiedSubmissionEvidence === true,
steeringPriorSubmission: guard.steeringPriorSubmission ? structuredClone(guard.steeringPriorSubmission) : null,
steeringPriorReadAfterCorrection: guard.steeringPriorReadAfterCorrection === true,
+ steeringHistoryInMemory: guard.steeringHistoryInMemory === true,
steeringMutationReceipts: structuredClone(guard.steeringMutationReceipts || []),
steeringPriorContext: structuredClone(guard.steeringPriorContext || null),
steeringPriorSubmissionReconciliation: structuredClone(guard.steeringPriorSubmissionReconciliation || null),
diff --git a/test/guards-steering-receipts.mjs b/test/guards-steering-receipts.mjs
index f5e8b7d35..85fb3d6a7 100644
--- a/test/guards-steering-receipts.mjs
+++ b/test/guards-steering-receipts.mjs
@@ -18,13 +18,19 @@ for (const build of ['chrome', 'firefox']) {
const xWorkflow = resolveAdapterWorkflowJob('https://x.com/', 'publish-post');
// A restart is a new Agent given the conversation as it was saved.
- async function fixture(task, { pageUrl = formUrl, gate: gateOverrides = {}, messages = null, tab: fixedTab = null } = {}) {
+ async function fixture(task, { pageUrl = formUrl, gate: gateOverrides = {}, saved = null, tab: fixedTab = null } = {}) {
const tab = fixedTab ?? nextTab++, options = { detachedRequestId: `guards-${tab}`, askStreamingEnabled: false };
const provider = { model: 'owned-offline', supportsVision: false, supportsTools: true, promptTier: 'full',
contextWindow: 128000, config: { category: 'local' }, chat() {} };
const agent = new Agent({ getActive: () => provider, getProvider: () => provider });
- agent.conversations.set(tab, messages ? JSON.parse(JSON.stringify(messages))
- : [{ role: 'system', content: 'System' }, { role: 'user', content: task }]);
+ if (saved) {
+ // Hydrate from the saved session entry, as the real code does.
+ const read = area.get;
+ area.get = async key => key === agent._convKey(tab) ? { [key]: JSON.parse(JSON.stringify(saved)) } : read(key);
+ try { await agent._hydrate(tab); } finally { area.get = read; }
+ } else {
+ agent.conversations.set(tab, [{ role: 'system', content: 'System' }, { role: 'user', content: task }]);
+ }
agent.conversationModes.set(tab, 'act');
agent._persist = () => {}; agent._persistNow = async () => ({ ok: true });
agent._ensureProgressSessionForCurrentTask = async () => {};
@@ -55,10 +61,15 @@ for (const build of ['chrome', 'firefox']) {
};
h.steeringBlock = (detected, name = 'click_ax', args = { ref_id: 'ref_9' }) => agent._steeringPriorSubmissionBlock(tab, name, args, detected);
h.close = () => { agent._finishSteeringRun(tab); agent._releaseRunEntry(tab); };
- // WebBrain restarted: only the saved conversation is left.
- h.restart = async (extra = []) => {
+ // WebBrain restarted: only what was saved with the conversation is left,
+ // here with any steps it took after the last save. An entry saved by an
+ // earlier version has no record of attempts.
+ h.restart = async (extra = [], { record = true } = {}) => {
+ const entry = agent._conversationStorageEntry(tab);
+ entry.messages = [...entry.messages, ...extra];
+ if (!record) delete entry.taskAttemptLedger;
h.close();
- return fixture(task, { pageUrl, gate: gateOverrides, tab, messages: [...agent.conversations.get(tab), ...extra] });
+ return fixture(task, { pageUrl, gate: gateOverrides, tab, saved: entry });
};
// A page read, the way the automatic screenshot after an action reads one.
h.read = (url = pageUrl) => agent._recordCompletionToolResult(tab, 'auto_screenshot', {},
@@ -238,6 +249,9 @@ for (const build of ['chrome', 'firefox']) {
const bskyPublish = (body, account = 'bluesky:alice.bsky.social') => ({ ...publish(snap(body, account)), url: 'https://bsky.app/', host: 'bsky.app' });
const newsletter = () => ({ ...formOn('https://newsletter.test/subscribe'), actionUrl: 'https://newsletter.test/subscribe/save',
fields: [{ label: 'Email', type: 'email', value: '[email protected]' }] });
+ // A Mastodon composer bound to its form action, as a corrected post on it is.
+ const mastodonForm = text => ({ ...mastodonCompose, actionUrl: 'https://mastoturk.org/api/v1/statuses',
+ fields: [{ label: 'Aklında ne var?', type: 'textarea', value: text }] });
const xSearch = () => ({ ...formOn('https://x.com/explore'), nonCommittingControl: true, method: 'GET',
fields: [{ label: 'Search', type: 'search', value: 'webbrain' }] });
// An action's result, recorded the way every action's result is.
@@ -340,8 +354,9 @@ for (const build of ['chrome', 'firefox']) {
}
});
- test(`${build}: a correction's receipt lists the platforms the task attempted, so a restart keeps them`, async () => {
- // The trace, with WebBrain restarted after the correction.
+ test(`${build}: what a task attempted is saved with the conversation, so a restart keeps it`, async () => {
+ // The trace, with WebBrain restarted after the correction, before or
+ // after other steps of the run.
for (const afterCorrection of ['nothing', 'tool steps']) {
const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://x.com/home' });
let r;
@@ -353,20 +368,16 @@ for (const build of ['chrome', 'firefox']) {
{ role: 'tool', tool_call_id: 'call_nav', content: '{"success":true}' }]);
const guard = r.agent._planExecutionGuards.get(r.tab);
assert.deepEqual(guard.steeringPriorSubmission?.taskAttemptPlatforms, ['mastoturk.org'], afterCorrection);
+ assert.deepEqual(r.agent._taskAttemptLedgers.get(r.tab).publications, [], afterCorrection);
assert.equal(guard.steeringPriorReadAfterCorrection, false, 'a restarted run reads again');
assert.equal((await r.check(xPublish('WebBrain test post', ''))).actionGuard.reason, 'prior_effect_unreconciled', afterCorrection);
r.read();
assert.equal(await r.check(xPublish('WebBrain test post', '')), null, afterCorrection);
- // Only steps after the correction leave the record incomplete, which
- // matters for the next correction.
attempt(r, xPublish('WebBrain test post'));
await r.steer('now on Bluesky too');
r.read();
const next = await r.check(bskyPublish('WebBrain test post', ''));
- if (afterCorrection === 'nothing') assert.equal(next, null, next?.error);
- else assert.equal(next?.actionGuard.reason, 'attempt_history_unknown', afterCorrection);
- // With both accounts read the platforms differ by account, as before.
- assert.equal(await r.check(bskyPublish('WebBrain test post')), null, afterCorrection);
+ assert.equal(next, null, `${afterCorrection}: ${next?.error}`);
} finally { (r || h).close(); }
}
// The X post and the newsletter, then "fix the typo in the tweet", then a
@@ -383,25 +394,169 @@ for (const build of ['chrome', 'firefox']) {
assert.equal((await r.check(xPublish('Hello world'))).actionGuard.reason, 'earlier_attempt_on_platform');
} finally { (r || h).close(); }
}
- // A conversation saved before the record existed: WebBrain cannot know,
- // so it says so once instead of looking for the Mastodon post on X.
- {
+ // Saved by an earlier version, without the record or the receipt's list:
+ // WebBrain cannot know, so it says so once instead of looking for the
+ // Mastodon post on X.
+ for (const afterCorrection of ['nothing', 'tool steps']) {
const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://x.com/home' });
let r;
try {
attempt(h, mastodonCompose, deliveredUnobserved);
await h.steer('ok now on X');
- const saved = h.agent.conversations.get(h.tab);
- delete saved.findLast(message => message.webbrainSteering).webbrainSteering.priorSubmission.taskAttemptPlatforms;
- r = await h.restart();
+ delete h.agent.conversations.get(h.tab).findLast(message => message.webbrainSteering).webbrainSteering.priorSubmission.taskAttemptPlatforms;
+ r = await h.restart(afterCorrection === 'nothing' ? [] : [
+ { role: 'assistant', content: '', tool_calls: [{ id: 'call_nav', type: 'function', function: { name: 'navigate', arguments: '{}' } }] },
+ { role: 'tool', tool_call_id: 'call_nav', content: '{"success":true}' }], { record: false });
r.read();
const blocked = await r.check(xPublish('WebBrain test post', ''));
- assert.equal(blocked.actionGuard.reason, 'attempt_history_unknown', blocked.error);
+ assert.equal(blocked.actionGuard.reason, 'attempt_history_unknown', `${afterCorrection}: ${blocked.error}`);
assert.equal(blocked.retryable, false);
} finally { (r || h).close(); }
}
});
+ // The trace again, with the X post sent under the correction and its
+ // result unknown, then the task continued: a run rebuilt from the
+ // conversation has no publication contract outcome or dispatch record of
+ // that post, so the task's record is what stops a second one.
+ test(`${build}: a post made under the correction holds a second one there in any later run`, async () => {
+ for (const resume of ['same run', 'Continue', 'typed continue', 'restart', 'restart, saved by an earlier version']) {
+ const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://x.com/home' });
+ let r = h;
+ try {
+ attempt(h, mastodonCompose, deliveredUnobserved);
+ await h.steer('ok now on X');
+ h.read();
+ assert.equal(await h.check(xPublish('WebBrain test post')), null);
+ attempt(h, xPublish('WebBrain test post'), deliveredUnobserved);
+ if (resume === 'Continue' || resume === 'typed continue') {
+ h.agent._storeContinuationExecutionEvidence(h.tab);
+ h.agent._beginCompletionInvariant(h.tab);
+ h.agent._startPlanExecutionGuard(h.tab, 'act', h.gate, resume === 'Continue' ? { trustedContinuation: true } : {});
+ } else if (resume.startsWith('restart')) {
+ r = await h.restart([], { record: resume === 'restart' });
+ }
+ assert.equal(r.agent._planExecutionGuards.get(r.tab).steeringHistoryInMemory === true, ['same run', 'Continue'].includes(resume), resume);
+ r.read();
+ const audited = r.audits.length;
+ const again = await r.check(xPublish('WebBrain test post'));
+ const another = await r.check(xPublish('WebBrain test post, take two'));
+ if (['same run', 'Continue'].includes(resume)) {
+ // The run that sent it still has its dispatch record; a different
+ // post is the publication contract's to refuse, as before.
+ assert.equal(again?.actionGuard.reason, 'duplicate_publication', resume);
+ } else {
+ const reason = resume === 'restart, saved by an earlier version' ? 'attempt_history_unknown' : 'earlier_attempt_on_platform';
+ assert.equal(again?.actionGuard.reason, reason, `${resume}: ${again?.error}`);
+ assert.equal(another?.actionGuard.reason, reason, resume);
+ assert.equal(r.audits.length, audited, `${resume}: neither reached authorization`);
+ }
+ } finally { r.close(); }
+ }
+ });
+
+ // Where the earlier attempt and the new post are told apart by account
+ // names, or the new one is an ordinary form such as a Mastodon composer,
+ // the earlier result still has to be observed as before; a page read
+ // under the correction does not stand in for it there.
+ test(`${build}: a page read under the correction counts only where the record clears the post`, async () => {
+ for (const variant of ['read in the run', 'search, then read', 'typed continue']) {
+ for (const kind of ['bluesky', 'mastodon']) {
+ const h = await fixture('Post "Hello wrold" on Bluesky and on X.', { pageUrl: kind === 'bluesky' ? 'https://bsky.app/' : 'https://mastoturk.org/publish' });
+ try {
+ if (kind === 'bluesky') attempt(h, bskyPublish('Hello wrold'), deliveredUnobserved);
+ else attempt(h, mastodonForm('Hello wrold'), deliveredUnobserved);
+ attempt(h, xPublish('Hello wrold'), deliveredUnobserved);
+ await h.steer(`fix the typo in the ${kind === 'bluesky' ? 'Bluesky' : 'Mastodon'} post`);
+ if (variant === 'search, then read') attempt(h, xSearch());
+ if (variant === 'typed continue') {
+ h.agent._beginCompletionInvariant(h.tab);
+ h.agent._startPlanExecutionGuard(h.tab, 'act', h.gate, {});
+ }
+ h.read();
+ const corrected = kind === 'bluesky' ? bskyPublish('Hello world') : mastodonForm('Hello world');
+ const blocked = await h.check(corrected);
+ const label = `${kind}, ${variant}`;
+ // The live receipt read in the run is the earlier result observed.
+ if (variant === 'read in the run') assert.equal(blocked, null, `${label}: ${blocked?.error}`);
+ else {
+ assert.equal(blocked?.actionGuard.reason, 'prior_effect_unreconciled', label);
+ assert.match(blocked.error, /Read the page that shows its result/, label);
+ }
+ } finally { h.close(); }
+ }
+ }
+ });
+
+ // Stop pressed while the X Post click was on its way: the tool's result
+ // never arrived, but the click may have posted.
+ test(`${build}: an action Stop interrupted after it was sent joins the record`, async () => {
+ const h = await fixture('Post "WebBrain test post" on X, then on mastoturk.org.', { pageUrl: 'https://x.com/home' });
+ try {
+ const { agent, tab } = h;
+ Object.assign(agent, { _skipPermissionGate: true, _ensureGateSetting: async () => true, _chromeProtectedPageFailure: async () => null,
+ _captchaMutationPreflight: async () => null, _adoptLiveSocialPublishWorkflow: async () => false, _workflowPreSubmitDispatchBlock: async () => null,
+ _messageRecipientGuardBlock: async () => null, _captureFormValidationState: async () => [], _withContentActionDeadline: async fn => fn(new AbortController().signal),
+ _preflightRichTextToolbarTarget: async () => ({ block: null }), _socialPublicationPreSubmitBlock: async () => null,
+ _auditRichTextToolbarTarget: async () => {}, _shouldAutoScreenshot: () => false });
+ h.probe = xPublish('WebBrain test post');
+ agent.executeTool = async (_tab, _name, _args, _update, context) => {
+ context._contentActionDispatchState.started = true;
+ agent.abort(tab);
+ throw context._contentActionAbortSignal.reason;
+ };
+ const messages = agent.conversations.get(tab);
+ const call = { id: 'call_post_x', type: 'function', function: { name: 'click_ax', arguments: '{"ref_id":"ref_9"}' } };
+ messages.push({ role: 'assistant', content: '', tool_calls: [call] });
+ await assert.rejects(agent._executeToolBatch(tab, [call], messages, () => {}, h.provider, null, new Set(['click_ax']), 1,
+ { beforeConsequentialTool: async () => true }), error => error.name === 'AbortError');
+ assert.deepEqual(agent._taskAttemptLedgers.get(tab).publications, ['twitter']);
+ // The task goes on: a Mastodon post, then "ok now on X".
+ agent._runAbortStates?.delete?.(tab);
+ h.close();
+ await agent._claimRunEntry(tab, 'interactive', h.options);
+ agent._beginSteeringRun(tab, () => {}, h.options);
+ agent._beginCompletionInvariant(tab);
+ agent._startPlanExecutionGuard(tab, 'act', h.gate, {});
+ attempt(h, mastodonCompose, deliveredUnobserved);
+ await h.steer('ok now on X');
+ h.read();
+ assert.equal((await h.check(xPublish('WebBrain test post'))).actionGuard.reason, 'earlier_attempt_on_platform');
+ } finally { h.close(); }
+ });
+
+ // Sent nothing, or committed nothing: neither is an attempt.
+ test(`${build}: a form the page rejected and a search before a correction are not attempts`, async () => {
+ for (const variant of ['rejected X post', 'search on X, then a correction']) {
+ const h = await fixture('Post "Hello" on X, then on mastoturk.org.', { pageUrl: 'https://x.com/home' });
+ try {
+ if (variant === 'rejected X post') attempt(h, xPublish('Hello'), { success: false, dispatched: true, formValidationFailed: true });
+ else { attempt(h, xSearch()); await h.steer('use the search results'); }
+ attempt(h, mastodonCompose, deliveredUnobserved);
+ await h.steer('ok now on X');
+ h.read();
+ const blocked = await h.check(xPublish('Hello'));
+ assert.equal(blocked, null, `${variant}: ${blocked?.error}`);
+ } finally { h.close(); }
+ }
+ });
+
+ // A deletion's receipt keeps its own rules; its guidance does not claim
+ // that no page was read.
+ test(`${build}: after a deletion on another site, the guidance does not blame a missing read`, async () => {
+ const h = await fixture('Delete my Mastodon post, then post on X.', { pageUrl: 'https://x.com/home' });
+ try {
+ h.agent._completionSubmitStates.set(h.tab, { dispatched: true, actionSequence: 4, mutationKind: 'delete',
+ originatingUrl: 'https://mastoturk.org/@alice/1', currentUrl: 'https://mastoturk.org/@alice', originatingDocument: 'm-doc',
+ actionResource: { complete: true, url: 'https://mastoturk.org/@alice/1' }, observedAfterSubmit: false });
+ await h.steer('ok now on X');
+ h.read();
+ const blocked = await h.check(xPublish('Hello'));
+ assert.equal(blocked?.actionGuard.reason, 'prior_effect_unreconciled');
+ assert.doesNotMatch(blocked.error, /no page was read/);
+ } finally { h.close(); }
+ });
+
// The trace's shape: the Mastodon post, "ok now on X", a page read, a search
// on X that replaces the live receipt, then the X post.
test(`${build}: a page read under the correction counts after a later receipt or run replaces the live one`, async () => {
diff --git a/test/quality/correction-cross-site-mv3.mjs b/test/quality/correction-cross-site-mv3.mjs
index e88435412..be039fedd 100644
--- a/test/quality/correction-cross-site-mv3.mjs
+++ b/test/quality/correction-cross-site-mv3.mjs
@@ -17,6 +17,11 @@ import { fixedChoiceQuestion, fixtureFixedChoiceReply, fixturePageTextReply } fr
// leave-prompt: WebBrain leaves a page with an unsent draft, and the
// page's "Leave site?" prompt is answered by WebBrain for its own
// navigation instead of waiting for the person.
+// restart-then-x: the run is stopped on X before posting, the extension's
+// worker restarts, and "continue" posts on X once: what the task attempted
+// was saved with the conversation, so WebBrain still knows X was not.
+// x-then-restart: the X post was sent, the worker restarts, and "continue"
+// tries to post it again: the saved record holds the second post.
// Pages, posts and the model are owned local fixtures; nothing leaves the host.
const require = createRequire(import.meta.url);
const library = (name, file) => fs.readFile(path.join(path.dirname(require.resolve(`${name}/package.json`)), file), 'utf8');
@@ -98,6 +103,58 @@ const variants = {
() => ({ name: 'done', args: { outcome: 'success', summary: 'Posted on mastoturk.org, then on X.' } }),
],
},
+ 'restart-then-x': {
+ before: [
+ () => ({ name: 'navigate', args: { url: 'https://mastoturk.org/publish' } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'set_field', args: { ref_id: refFrom(tree, /textbox "Aklında ne var\?"/), text: 'WebBrain test gönderisi', submit: false } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'click_ax', args: { ref_id: refFrom(tree, /button "Gönder"/) } }),
+ async (_tree, run) => { await run.steer(); return { name: 'get_accessibility_tree', args: { filter: 'all' } }; },
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ ],
+ after: [
+ () => ({ name: 'navigate', args: { url: 'https://x.com/home' } }),
+ // The person presses Stop before anything is posted on X.
+ async (_tree, run) => { await run.stop(); return { name: 'get_accessibility_tree', args: { filter: 'all' } }; },
+ ],
+ again: [
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'type_ax', args: { ref_id: refFrom(tree, /textbox "Post text"/), text: 'WebBrain test post', clear: true } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'click_ax', args: { ref_id: refFrom(tree, /button "Post"/) } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ () => ({ name: 'done', args: { outcome: 'success', summary: 'Posted on X.' } }),
+ ],
+ },
+ 'x-then-restart': {
+ before: [
+ () => ({ name: 'navigate', args: { url: 'https://mastoturk.org/publish' } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'set_field', args: { ref_id: refFrom(tree, /textbox "Aklında ne var\?"/), text: 'WebBrain test gönderisi', submit: false } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'click_ax', args: { ref_id: refFrom(tree, /button "Gönder"/) } }),
+ async (_tree, run) => { await run.steer(); return { name: 'get_accessibility_tree', args: { filter: 'all' } }; },
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ ],
+ after: [
+ () => ({ name: 'navigate', args: { url: 'https://x.com/home' } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'type_ax', args: { ref_id: refFrom(tree, /textbox "Post text"/), text: 'WebBrain test post', clear: true } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'click_ax', args: { ref_id: refFrom(tree, /button "Post"/) } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ () => ({ name: 'done', args: { outcome: 'success', summary: 'Posted on mastoturk.org, then on X.' } }),
+ ],
+ // After the restart the model, unsure the post went out, posts it again.
+ again: [
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'type_ax', args: { ref_id: refFrom(tree, /textbox "Post text"/), text: 'WebBrain test post', clear: true } }),
+ () => ({ name: 'get_accessibility_tree', args: { filter: 'all' } }),
+ tree => ({ name: 'click_ax', args: { ref_id: refFrom(tree, /button "Post"/) } }),
+ () => ({ name: 'done', args: { outcome: 'partial', summary: 'The X post was not sent again.' } }),
+ ],
+ },
'leave-prompt': {
before: [
() => ({ name: 'navigate', args: { url: 'https://x.com/home' } }),
@@ -144,6 +201,9 @@ async function runVariant(variant, script) {
};
let context, page, panel, tabId = null;
const run = {
+ async stop() {
+ await panel.evaluate(tabId => chrome.runtime.sendMessage({ target: 'background', action: 'abort', tabId }), tabId);
+ },
async steer() {
for (let wait = 0; !mastodonPosts.length && wait < 50; wait++) await new Promise(resolve => setTimeout(resolve, 100));
assert.equal(mastodonPosts.length, 1, 'The Mastodon post was sent before the correction');
@@ -170,7 +230,7 @@ async function runVariant(variant, script) {
else if (fixturePageTextReply(input.messages)) completion = { role: 'assistant', content: fixturePageTextReply(input.messages) };
else if (fixedChoiceQuestion(input.messages)) {
completion = { role: 'assistant', content: /^- "mode_claim":/m.test(system) ? JSON.stringify({ choice: 'result', evidence: '' })
- : fixtureFixedChoiceReply(input.messages, { deletion_prerequisite: 'keep_previous' }) };
+ : fixtureFixedChoiceReply(input.messages, { deletion_prerequisite: 'keep_previous', user_turn_kind: 'continue' }) };
} else if (system.startsWith('Check one separately proposed consequential action')) {
const trusted = JSON.parse(textOf(input.messages[1].content)), proposed = objectFrom(input.messages[2].content);
completion = { role: 'assistant', content: JSON.stringify({ key: proposed.key, authorized: true,
@@ -192,6 +252,7 @@ async function runVariant(variant, script) {
// before it ran is asked again.
const tools = toolMessages(input.messages);
if (steered && phase.name === 'before' && JSON.stringify(input.messages).includes(correction)) Object.assign(phase, { name: 'after', index: 0, lastId: null });
+ else if (run.continued && phase.name === 'after') Object.assign(phase, { name: 'again', index: 0, lastId: null });
else if (phase.lastId && tools.some(tool => tool.id === phase.lastId)) phase.index++;
const steps = script[phase.name];
const tree = tools.findLast(tool => tool.name === 'get_accessibility_tree' && tool.result?.pageContent);
@@ -273,12 +334,42 @@ async function runVariant(variant, script) {
await page.goto('chrome://extensions/');
tabId = await settings.evaluate(async () => (await chrome.tabs.query({})).find(tab => tab.url.startsWith('chrome://extensions'))?.id);
assert.ok(tabId, 'The run starts on chrome://extensions');
- const result = await message('chat', { tabId, text: task, mode: 'act', requestId: 'owned-cross-site', locale: 'en' });
- const toolResults = (result.updates || []).filter(event => event.type === 'tool_result').map(event => ({ name: event.data?.name,
+ const resultsOf = result => (result.updates || []).filter(event => event.type === 'tool_result').map(event => ({ name: event.data?.name,
success: event.data?.result?.success, reason: event.data?.result?.actionGuard?.reason || event.data?.result?.reasonCode || null,
error: event.data?.result?.error || null }));
- const evidence = { variant, toolResults, mastodonPosts, xPosts, dialogs, harnessAnswered, steered, unknownPrompts, judgeCalls,
- blockedRemote, finalUrl: page.url(), done: result.updates?.findLast(event => event.type === 'tool_result' && event.data?.name === 'done')?.data?.result || null };
+ const result = await message('chat', { tabId, text: task, mode: 'act', requestId: 'owned-cross-site', locale: 'en' });
+ const toolResults = resultsOf(result);
+ let continuedResults = null, workerRestarted = null, xPostsBeforeRestart = null, continuedReply = null;
+ if (script.again) {
+ // The extension's own worker stops, as Chrome stops an idle one, and
+ // the person continues the task in the fresh worker.
+ xPostsBeforeRestart = xPosts.length;
+ const devtools = await context.newCDPSession(settings), versions = new Map();
+ devtools.on('ServiceWorker.workerVersionUpdated', ({ versions: updates }) => updates.forEach(version => versions.set(version.versionId, version)));
+ await devtools.send('ServiceWorker.enable');
+ let version; const deadline = Date.now() + 5000;
+ while (!version && Date.now() < deadline) {
+ version = [...versions.values()].find(item => item.runningStatus === 'running' && item.scriptURL.startsWith(`${extensionOrigin}/`));
+ if (!version) await new Promise(resolve => setTimeout(resolve, 50));
+ }
+ assert.ok(version, 'The extension worker is running');
+ const boot = await worker.evaluate(() => globalThis.__crossSiteBoot ??= crypto.randomUUID());
+ await devtools.send('ServiceWorker.stopWorker', { versionId: version.versionId });
+ await devtools.detach().catch(() => {});
+ await message('agent_run_state', { tabId });
+ const fresh = context.serviceWorkers().find(item => item.url().startsWith(`${extensionOrigin}/`));
+ workerRestarted = await fresh.evaluate(() => globalThis.__crossSiteBoot ?? null) !== boot;
+ assert.equal(workerRestarted, true, 'The worker actually restarted');
+ await fresh.evaluate(() => { crossSiteFixtureAgent.clarifyTimeoutSec = -1; });
+ run.continued = true;
+ const continued = await message('chat', { tabId, text: 'continue', mode: 'act', requestId: 'owned-cross-site-continue', locale: 'en' });
+
+ continuedResults = resultsOf(continued);
+ continuedReply = String(continued?.content || '').slice(0, 400);
+ }
+ const evidence = { variant, toolResults, continuedResults, continuedReply, workerRestarted, xPostsBeforeRestart, mastodonPosts, xPosts, dialogs,
+ harnessAnswered, steered, unknownPrompts, judgeCalls, blockedRemote, finalUrl: page.url(),
+ done: result.updates?.findLast(event => event.type === 'tool_result' && event.data?.name === 'done')?.data?.result || null };
await fs.writeFile(path.join(out, 'evidence.json'), JSON.stringify(evidence, null, 2));
assert.deepEqual(unknownPrompts, [], 'Every model question was one the fixture answers');
const held = toolResults.filter(item => item.reason === 'prior_effect_unreconciled');
@@ -290,6 +381,19 @@ async function runVariant(variant, script) {
const navigations = toolResults.filter(item => item.name === 'navigate');
assert.equal(navigations.length, 2, `No back and forth between the sites: ${JSON.stringify(navigations)}`);
assert.equal(harnessAnswered, 0);
+ } else if (variant === 'restart-then-x') {
+ assert.deepEqual(mastodonPosts, ['WebBrain test gönderisi'], 'One Mastodon post');
+ assert.equal(xPostsBeforeRestart, 0, 'Nothing was posted on X before the restart');
+ const stopped = [...toolResults, ...continuedResults].filter(item => ['prior_effect_unreconciled', 'attempt_history_unknown',
+ 'earlier_attempt_on_platform'].includes(item.reason));
+ assert.deepEqual(stopped, [], `After the restart the X post was not held: ${JSON.stringify(continuedResults)}`);
+ assert.deepEqual(xPosts, ['WebBrain test post'], `One X post after "continue": ${JSON.stringify(continuedResults)}`);
+ } else if (variant === 'x-then-restart') {
+ assert.deepEqual(mastodonPosts, ['WebBrain test gönderisi'], 'One Mastodon post');
+ assert.equal(xPostsBeforeRestart, 1, `The X post went out before the restart: ${JSON.stringify(toolResults)}`);
+ assert.deepEqual(xPosts, ['WebBrain test post'], `No second X post after "continue": ${JSON.stringify(continuedResults)}`);
+ const again = continuedResults.filter(item => item.name === 'click_ax');
+ assert.deepEqual(again.map(item => item.reason), ['earlier_attempt_on_platform'], JSON.stringify(continuedResults));
} else {
assert.ok(dialogs.includes('beforeunload'), `The X page asked before it was left: ${JSON.stringify(dialogs)}`);
assert.equal(harnessAnswered, 0, 'WebBrain answered the leave prompt for its own navigation; nobody else had to');
@@ -299,6 +403,7 @@ async function runVariant(variant, script) {
assert.deepEqual(xPosts, [], 'The draft was never posted');
}
console.log(JSON.stringify({ passed: true, installedMV3: true, variant, mastodonPosts: mastodonPosts.length, xPosts: xPosts.length,
+ ...(workerRestarted !== null ? { workerRestarted, xPostsBeforeRestart } : {}),
dialogs, harnessAnswered, navigations: toolResults.filter(item => item.name === 'navigate').length, judgeCalls, blockedRemote }));
} finally {
for (const settle of settleDialogs) await settle();