Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Save the attempt record before an action is sent; publications decide
Claude2fe381ce110e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2e03af1f8..1e1f7dd93 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,7 +7,7 @@ This changelog was generated from the repository Git history and release tags. V## [Unreleased]### Fixed-- Posting on another site after a correction: after a Mastodon post, "ok now on X" no longer leaves WebBrain going back and forth between the two sites, typing the X post again each time. It counted X as the same platform as Mastodon, because it reads account names only on X and Bluesky, and asked to find the Mastodon post on X itself, which can never happen. The site a post was made on now names its platform when no account name was read, and a post on another site goes ahead once a page has been read since the correction, provided the task made no earlier attempt on that site. What the task attempted is saved with the conversation, so this still holds when the task continues after WebBrain restarted, and a post already sent on X under the correction is not sent again by a later run of the task. When the task did attempt that site before, or WebBrain has no full record of its earlier attempts, it stops once and says so, instead of searching for a post it cannot find. A search, a click that sent nothing or a form the page rejected is not an attempt; a click that Stop interrupted after it was sent is. x.com and twitter.com are one platform.+- Posting on another site after a correction: after a Mastodon post, "ok now on X" no longer leaves WebBrain going back and forth between the two sites, typing the X post again each time. It counted X as the same platform as Mastodon, because it reads account names only on X and Bluesky, and asked to find the Mastodon post on X itself, which can never happen. The site a post was made on now names its platform when no account name was read, and a post on another site goes ahead once a page has been read since the correction, provided the task made no earlier post on that site. What the task attempted is saved with the conversation, and just before each action that may post or submit something, so this still holds when the task continues after WebBrain restarted, and a post already sent on X under the correction is not sent again by a later run of the task. When the task did post on that site before, or WebBrain has no full record of its earlier attempts, it stops once and says so, instead of searching for a post it cannot find. A search, a settings save or consent click on X, a click that sent nothing or a form the page rejected is not such a post; a click that Stop interrupted after it was sent counts. x.com and twitter.com are one platform.- Chrome: a page's "Leave site?" prompt no longer waits for you when WebBrain navigates away during a task that started on a page the debugger cannot attach to, such as chrome://extensions, or whose debugger connection dropped. WebBrain arms its dialog handling again at the next debugger attach and around each of its own navigations, and answers Leave for its own navigation as it already did on other runs. If you cancel the debugger, prompts stay yours to answer. Firefox offers extensions no way to answer these prompts.- Chat sites such as WhatsApp: a request that points at the open chat instead of naming the person, such as "send some test messages to this one" or "message this user", now sends there instead of asking "which contact, and what text?". The planner never sees the page, and its rules counted only words like "reply here" as the open chat, so it asked the same question every time without reading the page. WebBrain still reads the open chat's header itself and binds every send to that one contact, and stops if no single chat is open. The planner is also told it cannot see the page, no longer reads a note about a screenshot it is never given, and leaves the wording of described messages, such as test messages or a thank-you, to the agent. One request can now cover the open chat and someone named besides it, as in "test messages here, then thank Rustem Bey": the open chat needs no confirmation, the reviewed plan shows the named person, and that person is confirmed once on WebBrain's card before the first send to them. Words that refer back to someone named earlier still mean that person, and on mail sites the people named besides "reply here" are handled as before. On WhatsApp, where a contact is confirmed through its contact drawer, one run still confirms only one chat's contact, so the second chat of such a request needs a request of its own; the agent is now told so, instead of being sent to re-read the page and retry.- A subscription, registration or sign-in wall in front of a news article is now recognized in any language, by a model reading the wall's text once, instead of only when it used English phrases such as "Subscribe to continue reading". A Turkish "Okumaya devam etmek için abone olun" overlay now keeps WebBrain from reading the article hidden behind it, as an English one did, while an offer to subscribe beside text you can still read is not taken for a wall. A wall inside the article is found also after a long preview, and everything after the wall's text stays withheld; the preview before it stays readable, except the short passage just before the wall's text. If a page cannot be read again after its wall was recognized, the read fails rather than return what the wall hides. A wall whose number changes on every read, such as a countdown, stays a wall while its new text is read, and needs no more readings once two of its numbers were read as the wall, while a count that stops being a wall when its number changes reads as what it is once read; a wall whose other text keeps changing makes the read fail rather than leak, and the agent may read the page once more in case the wall went away.diff --git a/docs/architecture.md b/docs/architecture.mdindex 0d9fe052d..965fd7b38 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -102,21 +102,24 @@ on X and Bluesky only, name a post's platform; any other attempt, such as aMastodon post or a form, is placed by the site it was made on. Hosts that sharetheir last two labels count as one site, and x.com and twitter.com as oneplatform. A post on another site than the earlier attempt counts as another-platform only when the correction's receipt lists every site the task had-attempted and the post's site is not among them. The agent records, for the-task from its first run on, the site of every dispatched attempt that may have-changed something and of every publication among them (a search, a send the-browser refused or a form the page rejected is not one; an action Stop-interrupted after it was sent is). The record is saved with the conversation,-and each correction copies its sites into its receipt. A post made under the-correction is kept from repeating by the publication contract's outcomes and-dispatch records of the run that applied it; a run rebuilt from the saved+platform only when the correction's receipt lists every publication the task+had made and none was on the post's site; on X and Bluesky only a composer+publication posts, so a settings save or a consent click there is not one.+The agent records, for the task from its first run on, the site of every+dispatched attempt that may have changed something and of every publication+among them (a search, a send the browser refused or a form the page rejected+is not one; an action Stop interrupted after it was sent is). The record is+saved with the conversation, and on its own just before each action that may+commit something is sent, so a restart right after that action still counts+it; each correction copies the record into its receipt. A post made under+the correction is kept from repeating by the publication contract's outcomes+and dispatch records of the run that applied it; a run rebuilt from the savedconversation, as after a restart or a typed "continue", has neither, so therea publication anywhere in the task on the post's site holds it too. Without a-complete record, or with an earlier attempt on the post's own site, the post-stops once with a report instead of a search for the other site's post. For-such a post, any page read since the earlier attempt was sent, including one-after the correction, makes its outcome known; where account names or an+complete record, or with an earlier publication on the post's own site, the+post stops once with a report instead of a search for the other site's post.+For such a post, any page read since the earlier attempt was sent, including+one after the correction, makes its outcome known; where account names or anordinary form tell the targets apart, the earlier result must be observed asbefore. Whether a correctionasks to delete the earlier post or record first is a fixed-choice questiondiff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 1dd3cd90a..d3feac6bc 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -1068,6 +1068,7 @@ export class Agent extends LoopDetector {// tabId -> { key, platforms }: the platforms (sites) of every attempt the// current task dispatched, or platforms null when they are not all known.this._taskAttemptLedgers = new Map();+ this._taskAttemptCheckpoints = new Set(); // tabIds whose saved record lists an action still in flightthis._recentSubmitClicks = new Map(); // tabId -> recent submit click timestampsthis._formValidationBlocks = new Map(); // tabId -> validation state that must change before another submitthis._steeringRuns = new Map(); // tabId -> inbox owned by one interactive run@@ -5421,7 +5422,7 @@ export class Agent extends LoopDetector {// It lists every platform the task attempted up to it, null when not// all are known, so a later run, even after a restart, knows them....(priorSubmission ? { priorSubmission: { ...structuredClone(priorSubmission),- taskAttemptPlatforms: this._correctionAttemptPlatforms(tabId, messages, priorBinding, priorSubmission) } } : {}),+ ...this._correctionAttempts(tabId, messages, priorBinding, priorSubmission) } } : {}),} });onUpdate('steering_applied', item);this._traceSteeringRevision(tabId, 'steering_applied', { revision: run.revision, messageId: item.id });@@ -5507,7 +5508,7 @@ export class Agent extends LoopDetector {const guard = this._startPlanExecutionGuard(tabId, mode, gate, revisionOptions);this._resetCompletionForSteeringRevision(tabId);guard.steeringPriorSubmission = priorSubmission ? { ...structuredClone(priorSubmission),- taskAttemptPlatforms: this._correctionAttemptPlatforms(tabId, messages, this._activeTaskBinding(messages), priorSubmission) } : null;+ ...this._correctionAttempts(tabId, messages, this._activeTaskBinding(messages), priorSubmission) } : null;guard.steeringPriorReadAfterCorrection = false;// This run applied the correction: what it sends under it stays in its// publication contract and dispatch records.@@ -5840,8 +5841,8 @@ export class Agent extends LoopDetector {// WebBrain cannot know it. The record follows the task through its runs,// corrections and Continue, and is saved with the conversation. Without a// record, as for a conversation saved by an earlier version, only a task- // that has not run since its request is known, and of a correction with- // nothing run since it only the sites its receipt lists.+ // that has not run since its request is known, or a correction with+ // nothing run since it, by the sites its receipt lists._taskAttemptLedger(tabId, messages = this.conversations?.get(tabId) || [], binding = this._activeTaskBinding(messages)) {const key = this._taskAttemptLedgerKey(tabId, messages, binding);const ledgers = this._taskAttemptLedgers ||= new Map();@@ -5859,7 +5860,11 @@ export class Agent extends LoopDetector {}if (this._noTaskRunSince(messages, from)) {if (!prior) { platforms = []; publications = []; }- else if (Array.isArray(prior.taskAttemptPlatforms)) platforms = [...prior.taskAttemptPlatforms];+ else if (Array.isArray(prior.taskAttemptPlatforms)) {+ platforms = [...prior.taskAttemptPlatforms];+ // Every publication is among the attempts, so their list covers it.+ publications = [...(Array.isArray(prior.taskAttemptPublications) ? prior.taskAttemptPublications : prior.taskAttemptPlatforms)];+ }}}const created = { key, platforms, publications };@@ -5890,30 +5895,78 @@ export class Agent extends LoopDetector {publication = true;}}- if (keys.size) this._addTaskAttemptKeys(tabId, [...keys], publication);+ if (keys.size) {+ this._addTaskAttemptKeys(tabId, [...keys], publication);+ this._taskAttemptCheckpoints?.delete(tabId);+ } else {+ this._resaveTaskAttemptCheckpoint(tabId);+ }} catch {this._markTaskAttemptsUnknown(tabId);}}+ // What an action may commit, judged before it is sent: the sites it may+ // change and whether it may publish there, or null when it commits nothing.+ // A control without its own address acts on the page it is on.+ _potentialTaskAttempt(name, args, detected, pageUrl = '') {+ const network = isNetworkMutation(name, args) || name === 'chrome_web_store_publish';+ const commit = (detected?.isSubmit === true && !this._nonCommittingSubmission(detected))+ || name === 'execute_js' || this._formValidationActionLooksSubmit(name, args, null, detected);+ if (!commit && !network) return null;+ const keys = new Set(commit ? this._attemptPlatformKeys(detected?.publicationSnapshot?.account,+ [detected?.url || pageUrl, detected?.frameUrl]) : []);+ if (network) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'+ ? 'https://chromewebstore.google.com/' : args?.url) || '*');+ return { keys: [...keys], publication: network || detected?.publicationControl === true || name === 'execute_js' };+ }+// An action that may have been sent but whose result was never recorded,- // because Stop or an error interrupted it: its site joins the record when+ // because Stop or an error interrupted it: its sites join the record when// it may have committed something, and an unknown site makes it unknown.- _recordInterruptedTaskAttempt(tabId, name, args, detected) {+ _recordInterruptedTaskAttempt(tabId, name, args, detected, potential = undefined) {try {- const network = isNetworkMutation(name, args) || name === 'chrome_web_store_publish';- const commit = (detected?.isSubmit === true && !this._nonCommittingSubmission(detected))- || name === 'execute_js' || this._formValidationActionLooksSubmit(name, args, null, detected);- if (!commit && !network) return;- const keys = new Set(commit ? this._attemptPlatformKeys(detected?.publicationSnapshot?.account, [detected?.url, detected?.frameUrl]) : []);- if (network) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'- ? 'https://chromewebstore.google.com/' : args?.url) || '*');- this._addTaskAttemptKeys(tabId, [...keys], network || detected?.publicationControl === true || name === 'execute_js');+ const attempt = potential === undefined ? this._potentialTaskAttempt(name, args, detected) : potential;+ if (attempt) this._addTaskAttemptKeys(tabId, attempt.keys, attempt.publication);+ this._taskAttemptCheckpoints?.delete(tabId);} catch {this._markTaskAttemptsUnknown(tabId);}}+ // Before an action that may commit something is sent, the record is saved+ // with its sites as possibly attempted, so a restart right after it cannot+ // forget it. The conversation itself is saved only every so often.+ async _checkpointTaskAttempt(tabId, name, args, detected) {+ let potential = null;+ try {+ const pageUrl = detected?.url ? '' : await this._currentUrl(tabId).catch(() => '');+ potential = this._potentialTaskAttempt(name, args, detected, pageUrl);+ if (!potential) return potential;+ const record = this._persistedTaskAttemptLedger(tabId);+ if (!record) return potential;+ const add = (list, values) => Array.isArray(list) ? [...new Set([...list, ...values])].slice(0, 65) : null;+ const platforms = add(record.platforms, potential.keys);+ const publications = potential.publication ? add(record.publications, potential.keys) : record.publications;+ await chrome.storage.session.set({ [this._taskAttemptCheckpointKey(tabId)]: { ...record,+ platforms: platforms?.length > 64 ? null : platforms, publications: publications?.length > 64 ? null : publications } });+ this._taskAttemptCheckpoints.add(tabId);+ } catch {}+ return potential;+ }++ // The saved record again, from memory: after an action that was not sent,+ // so a restart does not count it.+ _resaveTaskAttemptCheckpoint(tabId) {+ if (!this._taskAttemptCheckpoints?.delete(tabId)) return;+ try {+ const record = this._persistedTaskAttemptLedger(tabId);+ if (record) void Promise.resolve(chrome.storage.session.set({ [this._taskAttemptCheckpointKey(tabId)]: record })).catch(() => {});+ } catch {}+ }++ _taskAttemptCheckpointKey(tabId) { return `agentAttempts:${tabId}`; }+_addTaskAttemptKeys(tabId, keys, publication) {const ledger = this._taskAttemptLedger(tabId);const add = (list, values) => {@@ -5940,27 +5993,45 @@ export class Agent extends LoopDetector {const entry = this._taskAttemptLedgers?.get(tabId);return entry?.key ? { key: entry.key,platforms: Array.isArray(entry.platforms) ? [...entry.platforms] : null,- publications: Array.isArray(entry.publications) ? [...entry.publications] : null } : null;+ publications: Array.isArray(entry.publications) ? [...entry.publications] : null, savedAt: Date.now() } : null;}- _restoreTaskAttemptLedger(tabId, saved) {+ // The record saved with the conversation, merged with the one saved before+ // the latest action that may have committed something: for the same task+ // every site either lists counts, and an unknown list stays unknown.+ _restoreTaskAttemptLedger(tabId, saved, checkpoint = null) {const list = value => Array.isArray(value) && value.length <= 64 && value.every(item => typeof item === 'string' && item.length <= 300)? [...value] : null;- if (!saved || typeof saved !== 'object' || typeof saved.key !== 'string' || !saved.key || saved.key.length > 300) return;- (this._taskAttemptLedgers ||= new Map()).set(tabId, { key: saved.key, platforms: list(saved.platforms), publications: list(saved.publications) });- }-- // The platforms of every attempt the task made up to a correction, its- // receipt included, or null when they are not all known.- _correctionAttemptPlatforms(tabId, messages, binding, receipt) {+ const valid = record => record && typeof record === 'object' && typeof record.key === 'string' && !!record.key+ && record.key.length <= 300 ? { key: record.key, platforms: list(record.platforms), publications: list(record.publications),+ savedAt: Number(record.savedAt) || 0 } : null;+ const a = valid(saved), b = valid(checkpoint);+ if (!a && !b) return;+ const union = (x, y) => Array.isArray(x) && Array.isArray(y) ? [...new Set([...x, ...y])] : null;+ const merged = a && b && a.key === b.key+ ? { key: a.key, platforms: union(a.platforms, b.platforms), publications: union(a.publications, b.publications) }+ : (!a || (b && b.savedAt > a.savedAt) ? b : a);+ (this._taskAttemptLedgers ||= new Map()).set(tabId, { key: merged.key, platforms: merged.platforms, publications: merged.publications });+ }++ // What the task attempted up to a correction, its receipt included: the+ // sites of every attempt and of every publication, each null when not all+ // are known. The receipt carries them.+ _correctionAttempts(tabId, messages, binding, receipt) {try {- const ledger = this._taskAttemptLedger(tabId, messages, binding).platforms;- if (!Array.isArray(ledger)) return null;+ const ledger = this._taskAttemptLedger(tabId, messages, binding);const committed = receipt && receipt.nonCommitting !== true && receipt.formValidationFailed !== true;- const platforms = [...new Set([...ledger, ...(committed ? this._receiptPlatformKeys(receipt) : [])])];- return platforms.length > 64 ? null : platforms;+ const keys = committed ? this._receiptPlatformKeys(receipt) : [];+ const published = committed && (receipt.mutationKind === 'publish' || !!receipt.publicationSnapshot+ || !!receipt.workflowBinding?.socialPublication?.snapshot || keys.includes('*'));+ const add = (list, values) => {+ if (!Array.isArray(list)) return null;+ const merged = [...new Set([...list, ...values])];+ return merged.length > 64 ? null : merged;+ };+ return { taskAttemptPlatforms: add(ledger.platforms, keys), taskAttemptPublications: add(ledger.publications, published ? keys : []) };} catch {- return null;+ return { taskAttemptPlatforms: null, taskAttemptPublications: null };}}@@ -6016,10 +6087,12 @@ export class Agent extends LoopDetector {const currentPlatforms = this._attemptPlatformKeys(detected.publicationSnapshot?.account, [detected.url, detected.frameUrl]);const otherSite = publication && !priorPlatforms.includes('*') && !currentPlatforms.includes('*')&& !currentPlatforms.some(platform => priorPlatforms.includes(platform));- // The receipt's list, checked against the X and Bluesky outcomes and the- // receipts archived at earlier corrections. A sent-nothing outcome is not- // an attempt; an attempt of unknown site may have been anywhere.- const attempted = Array.isArray(prior?.taskAttemptPlatforms) ? prior.taskAttemptPlatforms : null;+ // On X and Bluesky only a composer publication posts, so the earlier+ // publications are what such a post may repeat: the receipt's list,+ // checked against the X and Bluesky outcomes and the receipts archived at+ // earlier corrections. A sent-nothing outcome is not an attempt; an+ // attempt of unknown site may have been anywhere.+ const attempted = Array.isArray(prior?.taskAttemptPublications) ? prior.taskAttemptPublications : null;const outcomePlatforms = archive => (archive?.contract?.actions || []).filter(action => action?.id && archive.outcomes?.[action.id] && archive.outcomes[action.id].status !== 'failed').map(action => action.platform || '*');@@ -6033,7 +6106,11 @@ export class Agent extends LoopDetector {...[guard.steeringPriorPublication, guard.steeringPriorContext?.socialPublication].flatMap(outcomePlatforms),...(guard.steeringMutationReceipts || []).filter(receipt => receipt?.dispatched === true&& receipt.nonCommitting !== true && receipt.formValidationFailed !== true)- .flatMap(receipt => this._receiptPlatformKeys(receipt))];+ .flatMap(receipt => {+ const keys = this._receiptPlatformKeys(receipt);+ return receipt.mutationKind === 'publish' || receipt.publicationSnapshot || receipt.workflowBinding?.socialPublication?.snapshot+ || keys.includes('*') ? keys : [];+ })];const earlierAttemptHere = earlierPlatforms.some(platform => currentPlatforms.includes(platform));const historyKnown = !!attempted && Array.isArray(sinceCorrection) && !earlierPlatforms.includes('*');const siteCrossPlatform = otherSite && historyKnown && !earlierAttemptHere;@@ -13228,11 +13305,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// An action that was sent, or may have been, before Stop or an error// cut its bookkeeping short still joins the task's record of attempts.const interrupted = cancellationState.active;- if (interrupted?.invoked && !interrupted.attemptRecorded) {+ if (interrupted && !interrupted.attemptRecorded) {const raw = interrupted.result || interrupted.dispatchState?.rawToolResult;- const unsent = raw?.noDispatch === true || raw?.dispatched === false+ const unsent = !interrupted.invoked || raw?.noDispatch === true || raw?.dispatched === false|| (!raw && interrupted.dispatchState?.tracksTextMutationDispatch === true && interrupted.dispatchState.started !== true);- if (!unsent) this._recordInterruptedTaskAttempt(tabId, interrupted.name, interrupted.args, interrupted.detected);+ if (!unsent) {+ this._recordInterruptedTaskAttempt(tabId, interrupted.name, interrupted.args, interrupted.detected,+ Object.hasOwn(interrupted, 'potentialAttempt') ? interrupted.potentialAttempt : undefined);+ } else {+ this._resaveTaskAttemptCheckpoint(tabId);+ }}if (this._checkAbort(tabId) || error?.name === 'AbortError') {// Repair the whole assistant tool batch before any caller persists a@@ -14662,8 +14744,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst genericChatBlock = pipelineToolbarPreflight.block || socialDispatchBlock ? null: await this._prepareGenericChatDispatch(tabId, fnName, fnArgs, messageRecipientExecutionContext, abortSignal);this._throwIfAborted(abortSignal);- if (!pipelineToolbarPreflight.block && !socialDispatchBlock && !genericChatBlock- && (this._hasPendingSteering(tabId) || await this._refreshPageFeedbackForCall(tabId, tc, onUpdate))) return { steered: true };+ if (!pipelineToolbarPreflight.block && !socialDispatchBlock && !genericChatBlock) {+ // Save the task's record with this action's sites before it can be+ // sent, ahead of the last correction check.+ callState.potentialAttempt = await this._checkpointTaskAttempt(tabId, fnName, fnArgs, detectedSubmitAction);+ this._throwIfAborted(abortSignal);+ if (this._hasPendingSteering(tabId) || await this._refreshPageFeedbackForCall(tabId, tc, onUpdate)) {+ this._resaveTaskAttemptCheckpoint(tabId);+ return { steered: true };+ }+ }const calculationDispatchBlock = this._pureCalculationToolBlock(tabId, fnName, fnArgs, calculationRequestScope);if (!pipelineToolbarPreflight.block && !socialDispatchBlock && !genericChatBlock && !calculationDispatchBlock) {if (detectedSubmitAction?.isSubmit === true && detectedSubmitAction.publicationControl === true) {@@ -18668,9 +18758,11 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dtry {const key = this._convKey(tabId);const cloudflareKey = cloudflareManagedChallengeStorageKey(tabId);- const [stored, cloudflareStored] = await Promise.all([+ const attemptKey = this._taskAttemptCheckpointKey(tabId);+ const [stored, cloudflareStored, attemptStored] = await Promise.all([chrome.storage.session.get(key),chrome.storage.session.get(cloudflareKey),+ Promise.resolve(chrome.storage.session.get(attemptKey)).catch(() => null),]);const cloudflareSignal = normalizeCloudflareManagedChallengeState(cloudflareStored?.[cloudflareKey],@@ -18785,7 +18877,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d});}this._restorePersistedRichTextToolbarAudit(tabId, entry.richTextToolbarAudit);- this._restoreTaskAttemptLedger(tabId, entry.taskAttemptLedger);+ this._restoreTaskAttemptLedger(tabId, entry.taskAttemptLedger, attemptStored?.[attemptKey]);const savedDispatch = entry.nativeCaptchaDispatch;const rootDocument = Array.isArray(savedDispatch?.documents)? savedDispatch.documents.find(document => document?.frameId === 0) : null;@@ -32523,6 +32615,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst t = this.persistTimers.get(tabId);if (t) { clearTimeout(t); this.persistTimers.delete(tabId); }try {+ this._taskAttemptCheckpoints?.delete(tabId);+ // The record saved before an action belongs to the cleared conversation.+ try { void Promise.resolve(chrome.storage.session.remove(this._taskAttemptCheckpointKey(tabId))).catch(() => {}); } catch {}const removed = chrome.storage.session.remove(this._convKey(tabId));if (this._nativeCaptchaSolutions?.has(tabId)) {return Promise.all([removed.then(() => this._persistNow(tabId)).catch(() => {}), steeringDiscarded]);diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex cc416339a..11e6c7cf2 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -772,6 +772,7 @@ export class Agent extends LoopDetector {// tabId -> { key, platforms }: the platforms (sites) of every attempt the// current task dispatched, or platforms null when they are not all known.this._taskAttemptLedgers = new Map();+ this._taskAttemptCheckpoints = new Set(); // tabIds whose saved record lists an action still in flightthis._recentSubmitClicks = new Map();this._formValidationBlocks = new Map(); // tabId -> validation state that must change before another submitthis._steeringRuns = new Map(); // tabId -> inbox owned by one interactive run@@ -5094,7 +5095,7 @@ export class Agent extends LoopDetector {// It lists every platform the task attempted up to it, null when not// all are known, so a later run, even after a restart, knows them....(priorSubmission ? { priorSubmission: { ...structuredClone(priorSubmission),- taskAttemptPlatforms: this._correctionAttemptPlatforms(tabId, messages, priorBinding, priorSubmission) } } : {}),+ ...this._correctionAttempts(tabId, messages, priorBinding, priorSubmission) } } : {}),} });onUpdate('steering_applied', item);this._traceSteeringRevision(tabId, 'steering_applied', { revision: run.revision, messageId: item.id });@@ -5180,7 +5181,7 @@ export class Agent extends LoopDetector {const guard = this._startPlanExecutionGuard(tabId, mode, gate, revisionOptions);this._resetCompletionForSteeringRevision(tabId);guard.steeringPriorSubmission = priorSubmission ? { ...structuredClone(priorSubmission),- taskAttemptPlatforms: this._correctionAttemptPlatforms(tabId, messages, this._activeTaskBinding(messages), priorSubmission) } : null;+ ...this._correctionAttempts(tabId, messages, this._activeTaskBinding(messages), priorSubmission) } : null;guard.steeringPriorReadAfterCorrection = false;// This run applied the correction: what it sends under it stays in its// publication contract and dispatch records.@@ -5513,8 +5514,8 @@ export class Agent extends LoopDetector {// WebBrain cannot know it. The record follows the task through its runs,// corrections and Continue, and is saved with the conversation. Without a// record, as for a conversation saved by an earlier version, only a task- // that has not run since its request is known, and of a correction with- // nothing run since it only the sites its receipt lists.+ // that has not run since its request is known, or a correction with+ // nothing run since it, by the sites its receipt lists._taskAttemptLedger(tabId, messages = this.conversations?.get(tabId) || [], binding = this._activeTaskBinding(messages)) {const key = this._taskAttemptLedgerKey(tabId, messages, binding);const ledgers = this._taskAttemptLedgers ||= new Map();@@ -5532,7 +5533,11 @@ export class Agent extends LoopDetector {}if (this._noTaskRunSince(messages, from)) {if (!prior) { platforms = []; publications = []; }- else if (Array.isArray(prior.taskAttemptPlatforms)) platforms = [...prior.taskAttemptPlatforms];+ else if (Array.isArray(prior.taskAttemptPlatforms)) {+ platforms = [...prior.taskAttemptPlatforms];+ // Every publication is among the attempts, so their list covers it.+ publications = [...(Array.isArray(prior.taskAttemptPublications) ? prior.taskAttemptPublications : prior.taskAttemptPlatforms)];+ }}}const created = { key, platforms, publications };@@ -5563,30 +5568,78 @@ export class Agent extends LoopDetector {publication = true;}}- if (keys.size) this._addTaskAttemptKeys(tabId, [...keys], publication);+ if (keys.size) {+ this._addTaskAttemptKeys(tabId, [...keys], publication);+ this._taskAttemptCheckpoints?.delete(tabId);+ } else {+ this._resaveTaskAttemptCheckpoint(tabId);+ }} catch {this._markTaskAttemptsUnknown(tabId);}}+ // What an action may commit, judged before it is sent: the sites it may+ // change and whether it may publish there, or null when it commits nothing.+ // A control without its own address acts on the page it is on.+ _potentialTaskAttempt(name, args, detected, pageUrl = '') {+ const network = isNetworkMutation(name, args) || name === 'chrome_web_store_publish';+ const commit = (detected?.isSubmit === true && !this._nonCommittingSubmission(detected))+ || name === 'execute_js' || this._formValidationActionLooksSubmit(name, args, null, detected);+ if (!commit && !network) return null;+ const keys = new Set(commit ? this._attemptPlatformKeys(detected?.publicationSnapshot?.account,+ [detected?.url || pageUrl, detected?.frameUrl]) : []);+ if (network) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'+ ? 'https://chromewebstore.google.com/' : args?.url) || '*');+ return { keys: [...keys], publication: network || detected?.publicationControl === true || name === 'execute_js' };+ }+// An action that may have been sent but whose result was never recorded,- // because Stop or an error interrupted it: its site joins the record when+ // because Stop or an error interrupted it: its sites join the record when// it may have committed something, and an unknown site makes it unknown.- _recordInterruptedTaskAttempt(tabId, name, args, detected) {+ _recordInterruptedTaskAttempt(tabId, name, args, detected, potential = undefined) {try {- const network = isNetworkMutation(name, args) || name === 'chrome_web_store_publish';- const commit = (detected?.isSubmit === true && !this._nonCommittingSubmission(detected))- || name === 'execute_js' || this._formValidationActionLooksSubmit(name, args, null, detected);- if (!commit && !network) return;- const keys = new Set(commit ? this._attemptPlatformKeys(detected?.publicationSnapshot?.account, [detected?.url, detected?.frameUrl]) : []);- if (network) keys.add(this._publicationSiteOf(name === 'chrome_web_store_publish'- ? 'https://chromewebstore.google.com/' : args?.url) || '*');- this._addTaskAttemptKeys(tabId, [...keys], network || detected?.publicationControl === true || name === 'execute_js');+ const attempt = potential === undefined ? this._potentialTaskAttempt(name, args, detected) : potential;+ if (attempt) this._addTaskAttemptKeys(tabId, attempt.keys, attempt.publication);+ this._taskAttemptCheckpoints?.delete(tabId);} catch {this._markTaskAttemptsUnknown(tabId);}}+ // Before an action that may commit something is sent, the record is saved+ // with its sites as possibly attempted, so a restart right after it cannot+ // forget it. The conversation itself is saved only every so often.+ async _checkpointTaskAttempt(tabId, name, args, detected) {+ let potential = null;+ try {+ const pageUrl = detected?.url ? '' : await this._currentUrl(tabId).catch(() => '');+ potential = this._potentialTaskAttempt(name, args, detected, pageUrl);+ if (!potential) return potential;+ const record = this._persistedTaskAttemptLedger(tabId);+ if (!record) return potential;+ const add = (list, values) => Array.isArray(list) ? [...new Set([...list, ...values])].slice(0, 65) : null;+ const platforms = add(record.platforms, potential.keys);+ const publications = potential.publication ? add(record.publications, potential.keys) : record.publications;+ await browser.storage.session.set({ [this._taskAttemptCheckpointKey(tabId)]: { ...record,+ platforms: platforms?.length > 64 ? null : platforms, publications: publications?.length > 64 ? null : publications } });+ this._taskAttemptCheckpoints.add(tabId);+ } catch {}+ return potential;+ }++ // The saved record again, from memory: after an action that was not sent,+ // so a restart does not count it.+ _resaveTaskAttemptCheckpoint(tabId) {+ if (!this._taskAttemptCheckpoints?.delete(tabId)) return;+ try {+ const record = this._persistedTaskAttemptLedger(tabId);+ if (record) void Promise.resolve(browser.storage.session.set({ [this._taskAttemptCheckpointKey(tabId)]: record })).catch(() => {});+ } catch {}+ }++ _taskAttemptCheckpointKey(tabId) { return `agentAttempts:${tabId}`; }+_addTaskAttemptKeys(tabId, keys, publication) {const ledger = this._taskAttemptLedger(tabId);const add = (list, values) => {@@ -5613,27 +5666,45 @@ export class Agent extends LoopDetector {const entry = this._taskAttemptLedgers?.get(tabId);return entry?.key ? { key: entry.key,platforms: Array.isArray(entry.platforms) ? [...entry.platforms] : null,- publications: Array.isArray(entry.publications) ? [...entry.publications] : null } : null;+ publications: Array.isArray(entry.publications) ? [...entry.publications] : null, savedAt: Date.now() } : null;}- _restoreTaskAttemptLedger(tabId, saved) {+ // The record saved with the conversation, merged with the one saved before+ // the latest action that may have committed something: for the same task+ // every site either lists counts, and an unknown list stays unknown.+ _restoreTaskAttemptLedger(tabId, saved, checkpoint = null) {const list = value => Array.isArray(value) && value.length <= 64 && value.every(item => typeof item === 'string' && item.length <= 300)? [...value] : null;- if (!saved || typeof saved !== 'object' || typeof saved.key !== 'string' || !saved.key || saved.key.length > 300) return;- (this._taskAttemptLedgers ||= new Map()).set(tabId, { key: saved.key, platforms: list(saved.platforms), publications: list(saved.publications) });- }-- // The platforms of every attempt the task made up to a correction, its- // receipt included, or null when they are not all known.- _correctionAttemptPlatforms(tabId, messages, binding, receipt) {+ const valid = record => record && typeof record === 'object' && typeof record.key === 'string' && !!record.key+ && record.key.length <= 300 ? { key: record.key, platforms: list(record.platforms), publications: list(record.publications),+ savedAt: Number(record.savedAt) || 0 } : null;+ const a = valid(saved), b = valid(checkpoint);+ if (!a && !b) return;+ const union = (x, y) => Array.isArray(x) && Array.isArray(y) ? [...new Set([...x, ...y])] : null;+ const merged = a && b && a.key === b.key+ ? { key: a.key, platforms: union(a.platforms, b.platforms), publications: union(a.publications, b.publications) }+ : (!a || (b && b.savedAt > a.savedAt) ? b : a);+ (this._taskAttemptLedgers ||= new Map()).set(tabId, { key: merged.key, platforms: merged.platforms, publications: merged.publications });+ }++ // What the task attempted up to a correction, its receipt included: the+ // sites of every attempt and of every publication, each null when not all+ // are known. The receipt carries them.+ _correctionAttempts(tabId, messages, binding, receipt) {try {- const ledger = this._taskAttemptLedger(tabId, messages, binding).platforms;- if (!Array.isArray(ledger)) return null;+ const ledger = this._taskAttemptLedger(tabId, messages, binding);const committed = receipt && receipt.nonCommitting !== true && receipt.formValidationFailed !== true;- const platforms = [...new Set([...ledger, ...(committed ? this._receiptPlatformKeys(receipt) : [])])];- return platforms.length > 64 ? null : platforms;+ const keys = committed ? this._receiptPlatformKeys(receipt) : [];+ const published = committed && (receipt.mutationKind === 'publish' || !!receipt.publicationSnapshot+ || !!receipt.workflowBinding?.socialPublication?.snapshot || keys.includes('*'));+ const add = (list, values) => {+ if (!Array.isArray(list)) return null;+ const merged = [...new Set([...list, ...values])];+ return merged.length > 64 ? null : merged;+ };+ return { taskAttemptPlatforms: add(ledger.platforms, keys), taskAttemptPublications: add(ledger.publications, published ? keys : []) };} catch {- return null;+ return { taskAttemptPlatforms: null, taskAttemptPublications: null };}}@@ -5689,10 +5760,12 @@ export class Agent extends LoopDetector {const currentPlatforms = this._attemptPlatformKeys(detected.publicationSnapshot?.account, [detected.url, detected.frameUrl]);const otherSite = publication && !priorPlatforms.includes('*') && !currentPlatforms.includes('*')&& !currentPlatforms.some(platform => priorPlatforms.includes(platform));- // The receipt's list, checked against the X and Bluesky outcomes and the- // receipts archived at earlier corrections. A sent-nothing outcome is not- // an attempt; an attempt of unknown site may have been anywhere.- const attempted = Array.isArray(prior?.taskAttemptPlatforms) ? prior.taskAttemptPlatforms : null;+ // On X and Bluesky only a composer publication posts, so the earlier+ // publications are what such a post may repeat: the receipt's list,+ // checked against the X and Bluesky outcomes and the receipts archived at+ // earlier corrections. A sent-nothing outcome is not an attempt; an+ // attempt of unknown site may have been anywhere.+ const attempted = Array.isArray(prior?.taskAttemptPublications) ? prior.taskAttemptPublications : null;const outcomePlatforms = archive => (archive?.contract?.actions || []).filter(action => action?.id && archive.outcomes?.[action.id] && archive.outcomes[action.id].status !== 'failed').map(action => action.platform || '*');@@ -5706,7 +5779,11 @@ export class Agent extends LoopDetector {...[guard.steeringPriorPublication, guard.steeringPriorContext?.socialPublication].flatMap(outcomePlatforms),...(guard.steeringMutationReceipts || []).filter(receipt => receipt?.dispatched === true&& receipt.nonCommitting !== true && receipt.formValidationFailed !== true)- .flatMap(receipt => this._receiptPlatformKeys(receipt))];+ .flatMap(receipt => {+ const keys = this._receiptPlatformKeys(receipt);+ return receipt.mutationKind === 'publish' || receipt.publicationSnapshot || receipt.workflowBinding?.socialPublication?.snapshot+ || keys.includes('*') ? keys : [];+ })];const earlierAttemptHere = earlierPlatforms.some(platform => currentPlatforms.includes(platform));const historyKnown = !!attempted && Array.isArray(sinceCorrection) && !earlierPlatforms.includes('*');const siteCrossPlatform = otherSite && historyKnown && !earlierAttemptHere;@@ -6324,9 +6401,11 @@ export class Agent extends LoopDetector {try {const key = this._convKey(tabId);const cloudflareKey = cloudflareManagedChallengeStorageKey(tabId);- const [stored, cloudflareStored] = await Promise.all([+ const attemptKey = this._taskAttemptCheckpointKey(tabId);+ const [stored, cloudflareStored, attemptStored] = await Promise.all([browser.storage.session.get(key),browser.storage.session.get(cloudflareKey),+ Promise.resolve(browser.storage.session.get(attemptKey)).catch(() => null),]);const cloudflareSignal = normalizeCloudflareManagedChallengeState(cloudflareStored?.[cloudflareKey],@@ -6439,7 +6518,7 @@ export class Agent extends LoopDetector {});}this._restorePersistedRichTextToolbarAudit(tabId, entry.richTextToolbarAudit);- this._restoreTaskAttemptLedger(tabId, entry.taskAttemptLedger);+ this._restoreTaskAttemptLedger(tabId, entry.taskAttemptLedger, attemptStored?.[attemptKey]);const savedDispatch = entry.nativeCaptchaDispatch;const rootDocument = Array.isArray(savedDispatch?.documents)? savedDispatch.documents.find(document => document?.frameId === 0) : null;@@ -11870,11 +11949,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// An action that was sent, or may have been, before Stop or an error// cut its bookkeeping short still joins the task's record of attempts.const interrupted = cancellationState.active;- if (interrupted?.invoked && !interrupted.attemptRecorded) {+ if (interrupted && !interrupted.attemptRecorded) {const raw = interrupted.result || interrupted.dispatchState?.rawToolResult;- const unsent = raw?.noDispatch === true || raw?.dispatched === false+ const unsent = !interrupted.invoked || raw?.noDispatch === true || raw?.dispatched === false|| (!raw && interrupted.dispatchState?.tracksTextMutationDispatch === true && interrupted.dispatchState.started !== true);- if (!unsent) this._recordInterruptedTaskAttempt(tabId, interrupted.name, interrupted.args, interrupted.detected);+ if (!unsent) {+ this._recordInterruptedTaskAttempt(tabId, interrupted.name, interrupted.args, interrupted.detected,+ Object.hasOwn(interrupted, 'potentialAttempt') ? interrupted.potentialAttempt : undefined);+ } else {+ this._resaveTaskAttemptCheckpoint(tabId);+ }}if (this._checkAbort(tabId) || error?.name === 'AbortError') {// Repair the whole assistant tool batch before any caller persists a@@ -13219,8 +13303,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst genericChatBlock = pipelineToolbarPreflight.block || socialDispatchBlock ? null: await this._prepareGenericChatDispatch(tabId, fnName, fnArgs, messageRecipientExecutionContext, abortSignal);this._throwIfAborted(abortSignal);- if (!pipelineToolbarPreflight.block && !socialDispatchBlock && !genericChatBlock- && (this._hasPendingSteering(tabId) || await this._refreshPageFeedbackForCall(tabId, tc, onUpdate))) return { steered: true };+ if (!pipelineToolbarPreflight.block && !socialDispatchBlock && !genericChatBlock) {+ // Save the task's record with this action's sites before it can be+ // sent, ahead of the last correction check.+ callState.potentialAttempt = await this._checkpointTaskAttempt(tabId, fnName, fnArgs, detectedSubmitAction);+ this._throwIfAborted(abortSignal);+ if (this._hasPendingSteering(tabId) || await this._refreshPageFeedbackForCall(tabId, tc, onUpdate)) {+ this._resaveTaskAttemptCheckpoint(tabId);+ return { steered: true };+ }+ }const calculationDispatchBlock = this._pureCalculationToolBlock(tabId, fnName, fnArgs, calculationRequestScope);if (!pipelineToolbarPreflight.block && !socialDispatchBlock && !genericChatBlock && !calculationDispatchBlock) {if (detectedSubmitAction?.isSubmit === true && detectedSubmitAction.publicationControl === true) {@@ -28766,6 +28858,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst t = this.persistTimers.get(tabId);if (t) { clearTimeout(t); this.persistTimers.delete(tabId); }try {+ this._taskAttemptCheckpoints?.delete(tabId);+ // The record saved before an action belongs to the cleared conversation.+ try { void Promise.resolve(browser.storage.session.remove(this._taskAttemptCheckpointKey(tabId))).catch(() => {}); } catch {}const removed = browser.storage.session.remove(this._convKey(tabId));if (this._nativeCaptchaSolutions?.has(tabId)) {return Promise.all([removed.then(() => this._persistNow(tabId)).catch(() => {}), steeringDiscarded]);diff --git a/test/completion-steering-receipt-authority.mjs b/test/completion-steering-receipt-authority.mjsindex b0cd762e0..3efe17efa 100644--- a/test/completion-steering-receipt-authority.mjs+++ b/test/completion-steering-receipt-authority.mjs@@ -114,7 +114,7 @@ for (const build of ['chrome', 'firefox']) {const physical = agent._completionSubmitStates.get(tab), guard = agent._planExecutionGuards.get(tab);assert.equal(physical.dispatched, true); assert.equal(physical.taskKey, original.taskKey);// The correction adds only the list of platforms the task attempted.- assert.deepEqual(guard.steeringPriorSubmission, { ...original, taskAttemptPlatforms: ['example.test'] });+ assert.deepEqual(guard.steeringPriorSubmission, { ...original, taskAttemptPlatforms: ['example.test'], taskAttemptPublications: [] });assert.deepEqual(guard.steeringMutationReceipts, [original]);assert.equal(agent._currentCompletionSubmitState(tab), null);const evidence = agent._completionSubmissionEvidence(tab, { relevantFormCount: 1, successMessages: ['Saved successfully.'] }, url);@@ -132,7 +132,7 @@ for (const build of ['chrome', 'firefox']) {assert.equal(fresh.taskKey, guard.taskKey); assert.notEqual(fresh.taskKey, original.taskKey);assert.equal(agent._currentCompletionSubmitState(tab), fresh);assert.equal((await h.done()).done, true); assert.equal(guard.verifiedSubmissionEvidence, true);- assert.equal(h.savedValue, 'Revised'); assert.deepEqual(guard.steeringPriorSubmission, { ...original, taskAttemptPlatforms: ['example.test'] });+ assert.equal(h.savedValue, 'Revised'); assert.deepEqual(guard.steeringPriorSubmission, { ...original, taskAttemptPlatforms: ['example.test'], taskAttemptPublications: [] });assert.deepEqual(guard.steeringMutationReceipts, [original], 'A new save leaves the prior receipt available for reconciliation.');} finally { h.close(); }});diff --git a/test/guards-steering-receipts.mjs b/test/guards-steering-receipts.mjsindex 85fb3d6a7..55ac49179 100644--- a/test/guards-steering-receipts.mjs+++ b/test/guards-steering-receipts.mjs@@ -5,7 +5,13 @@ import { defaultDecide, installModelStub } from './semantic-decision-stub.mjs';// Offline actual-method fixture: browser transport, the page probe result and// the independent authorization reply are simulated; receipts, accepted// steering, revision guards, reconciliation and done use the real Agent.-const area = { get: async () => ({ systemOneEnabled: false }), set: async () => {}, remove: async () => {} };+// Session storage keeps what the record saves before an action; anything+// else reads as the fixture's settings.+const sessionStore = new Map();+const area = { get: async key => typeof key === 'string' && sessionStore.has(key) ? { [key]: structuredClone(sessionStore.get(key)) }+ : { systemOneEnabled: false },+ set: async values => { for (const [key, value] of Object.entries(values || {})) if (key.startsWith('agentAttempts:')) sessionStore.set(key, structuredClone(value)); },+ remove: async () => {} };globalThis.chrome = globalThis.browser = { storage: { local: area, session: area }, runtime: { getURL: v => `extension://fixture/${v}` } };const formUrl = 'https://owned.example.test/settings';const saveUrl = 'https://owned.example.test/settings/save';@@ -136,7 +142,8 @@ for (const build of ['chrome', 'firefox']) {assert.equal(h.steeringBlock(corrected).noDispatch, true, variant);}// Only the list of platforms the task attempted is added, at the correction.- assert.deepEqual(guard.steeringPriorSubmission, { ...archived, taskAttemptPlatforms: ['example.test'] }, 'the original receipt stays immutable');+ assert.deepEqual(guard.steeringPriorSubmission, { ...archived, taskAttemptPlatforms: ['example.test'], taskAttemptPublications: [] },+ 'the original receipt stays immutable');} finally { h.close(); }}});@@ -394,24 +401,32 @@ for (const build of ['chrome', 'firefox']) {assert.equal((await r.check(xPublish('Hello world'))).actionGuard.reason, 'earlier_attempt_on_platform');} finally { (r || h).close(); }}- // Saved by an earlier version, without the record or the receipt's list:- // WebBrain cannot know, so it says so once instead of looking for the- // Mastodon post on X.- for (const afterCorrection of ['nothing', 'tool steps']) {- const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://x.com/home' });- let r;- try {- attempt(h, mastodonCompose, deliveredUnobserved);- await h.steer('ok now on X');- delete h.agent.conversations.get(h.tab).findLast(message => message.webbrainSteering).webbrainSteering.priorSubmission.taskAttemptPlatforms;- r = await h.restart(afterCorrection === 'nothing' ? [] : [- { role: 'assistant', content: '', tool_calls: [{ id: 'call_nav', type: 'function', function: { name: 'navigate', arguments: '{}' } }] },- { role: 'tool', tool_call_id: 'call_nav', content: '{"success":true}' }], { record: false });- r.read();- const blocked = await r.check(xPublish('WebBrain test post', ''));- assert.equal(blocked.actionGuard.reason, 'attempt_history_unknown', `${afterCorrection}: ${blocked.error}`);- assert.equal(blocked.retryable, false);- } finally { (r || h).close(); }+ // Saved without the record: the receipt's lists cover the time before the+ // correction, and only a run with nothing since it can rely on them. Saved+ // by a version without the receipt's lists either, WebBrain cannot know,+ // so it says so once instead of looking for the Mastodon post on X.+ for (const saved of ['receipt lists', 'nothing']) {+ for (const afterCorrection of ['nothing', 'tool steps']) {+ const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://x.com/home' });+ let r;+ try {+ attempt(h, mastodonCompose, deliveredUnobserved);+ await h.steer('ok now on X');+ const receipt = h.agent.conversations.get(h.tab).findLast(message => message.webbrainSteering).webbrainSteering.priorSubmission;+ if (saved === 'nothing') { delete receipt.taskAttemptPlatforms; delete receipt.taskAttemptPublications; }+ r = await h.restart(afterCorrection === 'nothing' ? [] : [+ { role: 'assistant', content: '', tool_calls: [{ id: 'call_nav', type: 'function', function: { name: 'navigate', arguments: '{}' } }] },+ { role: 'tool', tool_call_id: 'call_nav', content: '{"success":true}' }], { record: false });+ r.read();+ const blocked = await r.check(xPublish('WebBrain test post', ''));+ const label = `${saved}, ${afterCorrection}`;+ if (saved === 'receipt lists' && afterCorrection === 'nothing') assert.equal(blocked, null, `${label}: ${blocked?.error}`);+ else {+ assert.equal(blocked?.actionGuard.reason, 'attempt_history_unknown', `${label}: ${blocked?.error}`);+ assert.equal(blocked.retryable, false);+ }+ } finally { (r || h).close(); }+ }}});@@ -434,7 +449,11 @@ for (const build of ['chrome', 'firefox']) {h.agent._beginCompletionInvariant(h.tab);h.agent._startPlanExecutionGuard(h.tab, 'act', h.gate, resume === 'Continue' ? { trustedContinuation: true } : {});} else if (resume.startsWith('restart')) {- r = await h.restart([], { record: resume === 'restart' });+ // The X post's own steps follow the correction in the conversation.+ r = await h.restart([+ { role: 'assistant', content: '', tool_calls: [{ id: 'call_post', type: 'function', function: { name: 'click_ax', arguments: '{"ref_id":"ref_9"}' } }] },+ { role: 'tool', tool_call_id: 'call_post', content: '{"success":false,"dispatched":true,"outcomeUnknown":true}' }],+ { record: resume === 'restart' });}assert.equal(r.agent._planExecutionGuards.get(r.tab).steeringHistoryInMemory === true, ['same run', 'Continue'].includes(resume), resume);r.read();@@ -541,6 +560,90 @@ for (const build of ['chrome', 'firefox']) {}});+ // The X click went out, and the worker stopped before the conversation was+ // saved again: the record saved just before the click still has it.+ test(`${build}: an action sent just before a restart counts even when the conversation was saved before it`, async () => {+ for (const outcome of ['sent', 'refused by the browser']) {+ const task = 'go to mastoturk.org and try to post something';+ const h = await fixture(task, { pageUrl: 'https://x.com/home' });+ let r;+ try {+ attempt(h, mastodonCompose, deliveredUnobserved);+ await h.steer('ok now on X');+ h.read();+ assert.equal(await h.check(xPublish('WebBrain test post')), null);+ const savedBefore = h.agent._conversationStorageEntry(h.tab);+ // As the action pipeline does right before it sends the click.+ const potential = await h.agent._checkpointTaskAttempt(h.tab, 'click_ax', { ref_id: 'ref_9' }, xPublish('WebBrain test post'));+ assert.deepEqual(potential, { keys: ['twitter'], publication: true });+ if (outcome !== 'sent') attempt(h, xPublish('WebBrain test post'), refused);+ h.close();+ r = await fixture(task, { pageUrl: 'https://x.com/home', tab: h.tab, saved: savedBefore });+ r.read();+ const again = await r.check(xPublish('WebBrain test post'));+ if (outcome === 'sent') assert.equal(again?.actionGuard.reason, 'earlier_attempt_on_platform', again?.error);+ else assert.equal(again, null, again?.error);+ } finally { (r || h).close(); }+ }+ });++ // The real action pipeline saves the record before it sends the click.+ test(`${build}: the action pipeline saves the record before a click that may post is sent`, async () => {+ const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://x.com/home' });+ try {+ const { agent, tab } = h;+ Object.assign(agent, { _skipPermissionGate: true, _ensureGateSetting: async () => true, _chromeProtectedPageFailure: async () => null,+ _captchaMutationPreflight: async () => null, _adoptLiveSocialPublishWorkflow: async () => false, _workflowPreSubmitDispatchBlock: async () => null,+ _messageRecipientGuardBlock: async () => null, _captureFormValidationState: async () => [], _withContentActionDeadline: async fn => fn(new AbortController().signal),+ _preflightRichTextToolbarTarget: async () => ({ block: null }), _socialPublicationPreSubmitBlock: async () => null,+ _auditRichTextToolbarTarget: async () => {}, _shouldAutoScreenshot: () => false, _resourceMutationPreSubmitBlock: async () => null,+ _steeringPriorSubmissionBlock: () => null });+ h.probe = xPublish('WebBrain test post');+ let savedAtSend = null;+ agent.executeTool = async () => {+ savedAtSend = structuredClone(sessionStore.get(`agentAttempts:${tab}`) || null);+ return { success: true, dispatched: true };+ };+ const messages = agent.conversations.get(tab);+ const call = { id: 'call_post_x', type: 'function', function: { name: 'click_ax', arguments: '{"ref_id":"ref_9"}' } };+ messages.push({ role: 'assistant', content: '', tool_calls: [call] });+ await agent._executeToolBatch(tab, [call], messages, () => {}, h.provider, null, new Set(['click_ax']), 1,+ { beforeConsequentialTool: async () => true });+ assert.deepEqual(savedAtSend?.publications, ['twitter'], 'saved before the click was sent');+ assert.deepEqual(agent._taskAttemptLedgers.get(tab).publications, ['twitter']);+ } finally { h.close(); }+ });++ // Only a composer publication posts on X: a settings save or a consent+ // click there earlier in the task is not a post that a new one repeats.+ test(`${build}: a commit on X that is not a post does not hold a later X post`, async () => {+ const h = await fixture('Turn off X ads personalization, then post on mastoturk.org.', { pageUrl: 'https://x.com/home' });+ try {+ attempt(h, formOn('https://x.com/settings/ads'));+ attempt(h, mastodonCompose, deliveredUnobserved);+ const guard = await h.steer('ok now on X');+ assert.deepEqual(guard.steeringPriorSubmission.taskAttemptPlatforms, ['twitter', 'mastoturk.org']);+ assert.deepEqual(guard.steeringPriorSubmission.taskAttemptPublications, []);+ h.read();+ assert.equal(await h.check(xPublish('WebBrain test post')), null);+ } finally { h.close(); }+ });++ // A script or key press has no address of its own; it acts on its page.+ test(`${build}: an interrupted action without an address of its own counts on its page`, async () => {+ const h = await fixture('go to mastoturk.org and try to post something', { pageUrl: 'https://mastoturk.org/publish' });+ try {+ const args = { code: "document.querySelector('form').requestSubmit()" };+ const potential = await h.agent._checkpointTaskAttempt(h.tab, 'execute_js', args, null);+ assert.deepEqual(potential, { keys: ['mastoturk.org'], publication: true });+ h.agent._recordInterruptedTaskAttempt(h.tab, 'execute_js', args, null, potential);+ assert.deepEqual(h.agent._taskAttemptLedgers.get(h.tab).publications, ['mastoturk.org']);+ await h.steer('ok now on X');+ h.read();+ assert.equal(await h.check(xPublish('WebBrain test post')), null);+ } finally { h.close(); }+ });+// A deletion's receipt keeps its own rules; its guidance does not claim// that no page was read.test(`${build}: after a deletion on another site, the guidance does not blame a missing read`, async () => {@@ -597,6 +700,8 @@ for (const build of ['chrome', 'firefox']) {'https://newsletter.test/thanks', 'https://newsletter.test/thanks', { success: true, status: 200 });let guard = await h.steer('fix the typo in the tweet');assert.deepEqual(guard.steeringPriorSubmission.taskAttemptPlatforms, ['twitter', 'newsletter.test', 'example.org']);+ // Of those, the X post and the request were publications; the form was not.+ assert.deepEqual(guard.steeringPriorSubmission.taskAttemptPublications, ['twitter', 'example.org']);assert.deepEqual(h.agent.conversations.get(h.tab).findLast(message => message.webbrainSteering).webbrainSteering.priorSubmission.taskAttemptPlatforms, ['twitter', 'newsletter.test', 'example.org'], 'the saved receipt lists them too');h.read();@@ -604,15 +709,18 @@ for (const build of ['chrome', 'firefox']) {attempt(h, bskyPublish('Hello world'));guard = await h.steer('and on Mastodon');assert.deepEqual(guard.steeringPriorSubmission.taskAttemptPlatforms, ['twitter', 'newsletter.test', 'example.org', 'bluesky']);+ assert.deepEqual(guard.steeringPriorSubmission.taskAttemptPublications, ['twitter', 'example.org', 'bluesky']);} finally { h.close(); }});test(`${build}: an over-long record counts as unknown, and receipts from two runs stay two`, async () => {const h = await fixture('Sign me up on every site.', { pageUrl: 'https://x.com/home' });try {- for (let index = 0; index < 65; index++) attempt(h, formOn(`https://site${index}.test/subscribe`));+ for (let index = 0; index < 65; index++) h.agent._recordCompletionSubmitAttempt(h.tab, null, 'fetch_url',+ { url: `https://site${index}.test/api/subscribe`, method: 'POST' }, 'https://x.com/home', 'https://x.com/home', { success: true, status: 201 });+ attempt(h, mastodonCompose, deliveredUnobserved);const guard = await h.steer('ok now on X');- assert.equal(guard.steeringPriorSubmission.taskAttemptPlatforms, null, 'past 64 sites the record is not kept');+ assert.equal(guard.steeringPriorSubmission.taskAttemptPublications, null, 'past 64 sites the record is not kept');h.read();assert.equal((await h.check(xPublish('Hello', ''))).actionGuard.reason, 'attempt_history_unknown');} finally { h.close(); }