Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

relaxed monitoring in retry

Barack Sokullu8434953df843
diff --git a/docs/page-feedback-monitor.md b/docs/page-feedback-monitor.md
index 6521c34dc..577741e19 100644
--- a/docs/page-feedback-monitor.md
+++ b/docs/page-feedback-monitor.md
@@ -197,6 +197,18 @@ Steer or grant authorization. Enter and Alt+Enter retain their existing behavior
its unverified fingerprint. Five unsuccessful attempts end with
`action_context_unverified`; real target, document and user changes keep their
existing invalidation behavior.
+- Its explicit Retry offers `action_monitoring_relaxed` for one accepted
+ continuation on the same exact task and document. Only the background's
+ authenticated monitor-state reply establishes the private grant. The retry
+ captures the currently observed original control and checks its node,
+ ancestor chain, parent/form, authored semantics, form destination, visibility
+ and actionability at preparation and dispatch. Passive DOM counters may
+ change; agent-owned focus/scroll and native input continuation remain allowed.
+ Target changes, navigation, human intervention, corrections and completion
+ expire the grant. Ordinary Continue cannot grant it. Failed startup restores
+ a fresh recovery option; repeated delivery of one request follows one run.
+ Permission, publication, recipient and duplicate-delivery checks remain
+ active. The UI keeps one failure explanation and adds the explicit retry note.
- `calculate` is independent of passive DOM changes. A genuine arithmetic-only
task can retain its terminal answer after an unavailable AX refresh only when
an actual successful literal-result receipt still matches the current task,
diff --git a/docs/trace-format-compatibility.md b/docs/trace-format-compatibility.md
index 3d5229d59..bea404287 100644
--- a/docs/trace-format-compatibility.md
+++ b/docs/trace-format-compatibility.md
@@ -16,7 +16,7 @@ WebBrain's trace data has three independent version layers:
Planner alignment and publication checks record their auxiliary model requests and responses, including `requestedMaxTokens`, `candidateAttempt`, finish/token diagnostics, and a finite `validationFailure` code. Technical repairs keep the candidate binding fixed and are distinct from a replacement plan after a valid semantic rejection. Metadata-only traces retain those diagnostics without prompts, replies, arbitrary JSON property names, or checker explanations; content-enabled traces follow the existing recording policy.
-Recoverable failures may include a `failure_recovery_offered` note with the failed stage and retry mode. Chat controls use an opaque runtime recovery ID bound to the exact task and conversation. Starting another run, changing the task, stopping, or choosing Give up expires the offer. Model, response-generation, context, and interaction failures resume through normal continuation checks; publication Retry is the separately bound, one-use audit exemption described in the publication contract.
+Recoverable failures may include a `failure_recovery_offered` note with the failed stage and retry mode. Chat controls use an opaque runtime recovery ID bound to the exact task and conversation. Starting another run, changing the task, stopping, or choosing Give up expires the offer. Model, response-generation and interaction failures resume through normal continuation checks. For `action_context_unverified`, `action_monitoring_relaxed` identifies one explicitly accepted Retry run on the same task and document; publication Retry remains the separately bound, one-use audit exemption described in the publication contract. Metadata-only `page_action_validation` notes include a finite `certificationFailure` or dispatch `predicate` and a `monitoringRelaxed` boolean. Private grant tokens and target/page content are excluded.
Tool events may include an optional `data.outcome` summary containing bounded dispatch, application, provider, error, and CAPTCHA-gate diagnostics. In lossless recordings, this summary survives the content-budget marker, while answer tokens, cookies, provider keys, and form inputs are excluded from the summary. Readers should still honor `_truncated` and `losslessBudgetOmitted`; the summary does not reconstruct discarded content.
diff --git a/package.json b/package.json
index e5d61bab5..2de66a17d 100644
--- a/package.json
+++ b/package.json
@@ -9,7 +9,8 @@
"test:authenticated": "npm --prefix website-tests test",
"test:authenticated:installed": "npm --prefix website-tests run test:installed",
"test": "npm run test:failure-recovery && npm run test:shard:quick && npm run test:shard:runtime-unit && npm run test:shard:browser-a && npm run test:shard:browser-b1 && npm run test:shard:browser-b2 && npm run test:shard:browser-c && npm run test:shard:tail",
- "test:failure-recovery": "node --test test/failure-recovery.mjs test/failure-recovery-ui.mjs test/post-submit-model-recovery.mjs test/run-ui-response-retry.mjs test/ask-stream-retry-ui.mjs",
+ "test:failure-recovery": "node --test test/failure-recovery.mjs test/failure-recovery-ui.mjs test/post-submit-model-recovery.mjs test/run-ui-response-retry.mjs test/ask-stream-retry-ui.mjs test/page-monitor-relaxed-retry.mjs && npm run test:quality:relaxed-monitoring-retry",
+ "test:quality:relaxed-monitoring-retry": "node test/quality/relaxed-monitoring-retry-mv3.mjs",
"test:captcha:ui": "node test/captcha-settings-ui.mjs && node test/captcha-application-ui.mjs",
"test:captcha:bridge:ui": "node test/captcha-callback-bridge-ui.mjs",
"test:billing-notice": "node test/billing-payment-notice.mjs",
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 00a126184..a660f3871 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -6816,6 +6816,12 @@ export class Agent extends LoopDetector {
// Claim synchronously before awaiting the external guard so teacher-mode
// startup cannot race a run whose persisted teacher-state check is pending.
this._runningTabs.add(tabId);
+ const monitoringRetry = runOptions.trustedContinuation === true
+ ? this._failureRecovery.takeMonitoringRetry(this._failureRecoveryBinding(tabId), String(runOptions.detachedRequestId || ''))
+ : null;
+ this._monitoringRetryStarts ??= new Map();
+ if (monitoringRetry) this._monitoringRetryStarts.set(tabId, monitoringRetry);
+ else this._monitoringRetryStarts.delete(tabId);
this._failureRecovery.clear(tabId);
this._jevSessions?.delete(tabId);
for (const [id, pending] of this._jevPendingCalls || []) if (pending.tabId === tabId) this._jevPendingCalls.delete(id);
@@ -6840,19 +6846,25 @@ export class Agent extends LoopDetector {
await this.assertRunStartAllowed(tabId, defaultKind, runOptions);
this._throwIfAborted(signal);
if (!this._isStandaloneChatRun(runOptions)) {
- await this._beginPageFeedbackRun(tabId, this._runEntryKind(defaultKind, runOptions), signal);
+ await this._beginPageFeedbackRun(tabId, this._runEntryKind(defaultKind, runOptions), signal, monitoringRetry);
}
this._throwIfAborted(signal);
const feedbackRun = this._pageFeedbackRuns?.get(tabId);
if (feedbackRun && typeof onUpdate === 'function') feedbackRun.onUpdate = onUpdate;
}, 'Browser startup', RUN_STARTUP_TIMEOUT_MS, controller.signal);
} catch (error) {
+ this._restoreFailureRecoveryStart(tabId, monitoringRetry?.requestId, onUpdate, monitoringRetry);
try { this._finishSteeringRun(tabId); } finally { this._releaseRunEntry(tabId); }
throw error;
}
}
_releaseRunEntry(tabId) {
+ const feedbackRun = this._pageFeedbackRuns?.get(tabId);
+ if (feedbackRun?.monitoringRetry && !feedbackRun.monitoringRetry.started) {
+ this._restoreFailureRecoveryStart(tabId, feedbackRun.monitoringRetry.requestId, feedbackRun.onUpdate, feedbackRun.monitoringRetry);
+ }
+ this._monitoringRetryStarts?.delete(tabId);
this._ownedNativeChatReadRepairs.delete(tabId);
this._finishPageFeedbackRun(tabId);
cdpClient.stopDialogHandling(tabId);
@@ -6901,6 +6913,7 @@ export class Agent extends LoopDetector {
}
const id = String(messageId);
if (!run.acceptedIds.has(id)) {
+ this._invalidateMonitoringRetry?.(tabId, 'task_changed');
run.acceptedIds.add(id);
run.messages.push({ id, text: text.trim() });
// A reviewed plan belongs to the task before this correction. Wake the
@@ -7393,12 +7406,18 @@ export class Agent extends LoopDetector {
if (this._checkAbort(tabId) || this._hasPendingSteering(tabId)) return null;
const binding = this._failureRecoveryBinding(tabId);
const recoveryId = 'recovery_' + secureRandomBase36Token(24);
+ const feedbackRun = this._pageFeedbackRuns?.get(tabId);
+ const monitoringPage = details.stage === 'action_preparation' && details.code === 'action_context_unverified'
+ && this._isActionMode(this._effectiveRunMode(tabId)) && feedbackRun?.frames.get(0)?.token
+ ? { url: feedbackRun.url, documentToken: feedbackRun.frames.get(0).token,
+ documentId: feedbackRun.documents.get(0)?.id || '' } : null;
const descriptor = { recoveryId, stage: details.stage || 'model',
code: details.code || 'request_failed', message: String(details.message || 'The task could not finish.').slice(0,2000),
- retryMode: details.stage === 'publication_audit' ? 'publication_without_audit' : 'resume',
+ retryMode: details.stage === 'publication_audit' ? 'publication_without_audit'
+ : monitoringPage ? 'action_monitoring_relaxed' : 'resume',
retryLabel: 'Retry', giveUpLabel: 'Give up',
...(details.diagnostics ? { diagnostics: String(details.diagnostics).slice(0,6000) } : {}) };
- this._failureRecovery.offer({ ...binding,
+ this._failureRecovery.offer({ ...binding, ...(monitoringPage ? { monitoringPage } : {}),
...(details.publicationKey ? { publicationKey: details.publicationKey,
publicationTaskKey: details.taskKey } : {}) }, descriptor);
onUpdate?.('failure_recovery', descriptor);
@@ -7482,13 +7501,33 @@ export class Agent extends LoopDetector {
return { content: details.message, status: postAction ? 'partial' : 'error' };
}
- requestFailureRecovery(tabId, recoveryId, choice) {
+ _restoreFailureRecoveryStart(tabId, requestId, onUpdate = null, retry = this._failureRecovery.monitoringRetries.get(tabId)) {
+ if (!retry || retry.requestId !== requestId || retry.started || retry.invalidated || retry.restored || this._checkAbort(tabId)) return false;
+ if (this._hasPendingSteering(tabId)) { retry.invalidated = true; return false; }
+ const current = this._failureRecoveryBinding(tabId), binding = retry.offer.binding;
+ if (!['conversationId', 'taskKey', 'taskSignature'].every(key => binding[key] === current[key])) return false;
+ retry.restored = true;
+ const descriptor = { ...retry.offer.descriptor, recoveryId: 'recovery_' + secureRandomBase36Token(24) };
+ this._failureRecovery.offer(binding, descriptor);
+ onUpdate?.('failure_recovery', descriptor);
+ return true;
+ }
+
+ requestFailureRecovery(tabId, recoveryId, choice, requestId = '') {
if (!['retry', 'give_up'].includes(choice)) return { ok: false, error: 'Invalid recovery choice.' };
if (this.isRunning(tabId)) return { ok: false, error: 'Wait for the current run to finish before choosing recovery.' };
const offer = this._failureRecovery.current(this._failureRecoveryBinding(tabId));
if (!offer || offer.descriptor.recoveryId !== recoveryId) {
return { ok: false, error: 'This retry has expired. Start or continue the current task.' };
}
+ if (choice === 'retry' && offer.descriptor.retryMode === 'action_monitoring_relaxed') {
+ if (typeof requestId !== 'string' || !requestId.trim() || requestId.length > 200
+ || !this._failureRecovery.reserveMonitoringRetry(this._failureRecoveryBinding(tabId), recoveryId,
+ requestId, secureRandomBase36Token(24))) {
+ return { ok: false, error: 'This recovery requires its own current retry request.' };
+ }
+ return { ok: true, continue: true, retryMode: offer.descriptor.retryMode };
+ }
if (choice === 'retry' && offer.descriptor.retryMode === 'publication_without_audit') {
const granted = this._grantSocialPublicationAuditRetry(tabId, { ...offer.binding,
taskKey: offer.binding.publicationTaskKey || offer.binding.taskKey });
@@ -18329,7 +18368,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
preparationStage = 'reprobe_after_pointer';
const current = await chrome.tabs.sendMessage(tabId, { target: 'content', action: 'click_ax',
params: { ...contentArgs, _prepareTrustedNativeButton: true, _requireTrustedNativeButton: true,
- expectedModelSnapshot: dispatchCall.expectedModelSnapshot, allowPassiveRebase: dispatchCall.allowPassiveRebase === true } }, messageOptions);
+ expectedModelSnapshot: dispatchCall.expectedModelSnapshot, allowPassiveRebase: dispatchCall.allowPassiveRebase === true,
+ ...(dispatchCall.relaxedMonitorGrantToken && this._monitoringRetryCurrent(tabId, run)?.grantToken === dispatchCall.relaxedMonitorGrantToken
+ ? { relaxedMonitorGrantToken: dispatchCall.relaxedMonitorGrantToken } : {}) } }, messageOptions);
assertCurrent();
if (current?.success !== true || current.trustedButtonRequired !== true) {
return failure(typeof current?.error === 'string' && current.error
@@ -18625,7 +18666,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const prepared = await chrome.tabs.sendMessage(tabId, { target: 'content', action: 'click_ax',
params: { ...contentArgs, _prepareTrustedNativeButton: true,
_requireTrustedNativeButton: false, expectedModelSnapshot: certifiedCall.expectedModelSnapshot,
- allowPassiveRebase: certifiedCall.allowPassiveRebase === true },
+ allowPassiveRebase: certifiedCall.allowPassiveRebase === true,
+ ...(certifiedCall.relaxedMonitorGrantToken && this._monitoringRetryCurrent(tabId, certifiedRun)?.grantToken === certifiedCall.relaxedMonitorGrantToken
+ ? { relaxedMonitorGrantToken: certifiedCall.relaxedMonitorGrantToken } : {}) },
...(actionDeadlineAt > 0 ? { actionDeadlineAt } : {}),
}, messageOptions);
throwIfAborted();
@@ -46336,6 +46379,13 @@ If the user has already named or confirmed this exact recipient, do NOT ask agai
const sendContent = message => chrome.tabs.sendMessage(tabId, {
...message, ...(actionDeadlineAt > 0 ? { actionDeadlineAt } : {}),
}, { frameId });
+ const monitoringPreparation = () => {
+ const run = this._pageFeedbackRuns?.get(tabId), call = run?.dispatchCall;
+ if (frameId !== 0 || !call?.expectedModelSnapshot || call.args?.ref_id !== args.ref_id
+ || !call.relaxedMonitorGrantToken || this._monitoringRetryCurrent(tabId, run)?.grantToken !== call.relaxedMonitorGrantToken) return {};
+ return { expectedModelSnapshot: call.expectedModelSnapshot,
+ relaxedMonitorGrantToken: call.relaxedMonitorGrantToken, allowPassiveRebase: true };
+ };
const finishPreparationFailure = async failure => {
if (!trustedDispatched || trustedTextAttempted || !beforePreparation?.documentToken || abortSignal?.aborted) return failure;
const afterPreparation = await this._textMutationValueDigest(tabId, digestTarget);
@@ -46363,7 +46413,7 @@ If the user has already named or confirmed this exact recipient, do NOT ask agai
let prepared = await sendContent({
target: 'content',
action: 'ax_prepare_field_for_trusted_type',
- params: { ref_id: args.ref_id, selectionMode: contentEditableSelectionMode },
+ params: { ref_id: args.ref_id, selectionMode: contentEditableSelectionMode, ...monitoringPreparation() },
});
this._throwIfAborted(abortSignal);
if (!prepared?.success) {
@@ -46402,7 +46452,7 @@ If the user has already named or confirmed this exact recipient, do NOT ask agai
const reselected = await sendContent({
target: 'content',
action: 'ax_prepare_field_for_trusted_type',
- params: { ref_id: args.ref_id, selectionMode: contentEditableSelectionMode },
+ params: { ref_id: args.ref_id, selectionMode: contentEditableSelectionMode, ...monitoringPreparation() },
});
this._throwIfAborted(abortSignal);
if (!reselected?.success) {
diff --git a/src/chrome/src/agent/failure-recovery.js b/src/chrome/src/agent/failure-recovery.js
index 820a6f479..8b50277a2 100644
--- a/src/chrome/src/agent/failure-recovery.js
+++ b/src/chrome/src/agent/failure-recovery.js
@@ -1,8 +1,9 @@
// Recovery choices come from the extension UI, never from a model tool call.
export class FailureRecoveryRegistry {
- constructor() { this.offers = new Map(); }
+ constructor() { this.offers = new Map(); this.monitoringRetries = new Map(); }
offer(binding, descriptor) {
+ this.monitoringRetries.delete(binding.tabId);
const offer = { binding: { ...binding }, descriptor: { ...descriptor } };
this.offers.set(binding.tabId, offer);
return { ...offer.descriptor };
@@ -14,12 +15,31 @@ export class FailureRecoveryRegistry {
if (offer.binding.conversationId !== binding.conversationId || offer.binding.taskKey !== binding.taskKey
|| offer.binding.taskSignature !== binding.taskSignature) {
this.offers.delete(binding.tabId);
+ this.monitoringRetries.delete(binding.tabId);
return null;
}
return offer;
}
- clear(tabId) { this.offers.delete(tabId); }
+ reserveMonitoringRetry(binding, recoveryId, requestId, grantToken) {
+ const offer = this.current(binding);
+ if (!requestId || !grantToken || offer?.descriptor.recoveryId !== recoveryId
+ || offer.descriptor.retryMode !== 'action_monitoring_relaxed' || !offer.binding.monitoringPage) return null;
+ const previous = this.monitoringRetries.get(binding.tabId);
+ if (previous) return previous.requestId === requestId ? previous : null;
+ const retry = { offer, requestId, grantToken };
+ this.monitoringRetries.set(binding.tabId, retry);
+ return retry;
+ }
+
+ takeMonitoringRetry(binding, requestId) {
+ const offer = this.current(binding), retry = this.monitoringRetries.get(binding.tabId);
+ if (!retry || retry.offer !== offer || retry.requestId !== requestId) return null;
+ this.clear(binding.tabId);
+ return retry;
+ }
+
+ clear(tabId) { this.offers.delete(tabId); this.monitoringRetries.delete(tabId); }
}
export function modelFailureRecovery(error, { postAction = false, phase = 'model' } = {}) {
diff --git a/src/chrome/src/agent/page-feedback.js b/src/chrome/src/agent/page-feedback.js
index f50dd6686..9dfc0366b 100644
--- a/src/chrome/src/agent/page-feedback.js
+++ b/src/chrome/src/agent/page-feedback.js
@@ -218,11 +218,13 @@ export async function beforePageAgentDispatch(api, tabId, details = {}) {
let acknowledgement;
let preparationAcknowledgement;
const expectedModelSnapshot = owner.expectedModelSnapshot?.();
+ const relaxedMonitorGrantToken = owner.relaxedMonitorGrantToken?.();
try {
if (!details.release && details.fenceOnly && prepareMonitor) {
preparationAcknowledgement = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_prepare',
params: { ...monitorParams, tool: dispatchDetails.tool || dispatchDetails.kind,
allowPassiveRebase: owner.allowPassiveRebase?.() === true,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}),
...(expectedModelSnapshot ? { expectedModelSnapshot } : {}) } }, { frameId: Number(details.frameId) || 0 });
}
acknowledgement = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_dispatch',
@@ -312,11 +314,39 @@ function sameOriginPath(expected, actual) {
}
export const pageFeedbackMethods = {
- async _beginPageFeedbackRun(tabId, kind, signal = null) {
+ _isPassiveMonitoringFeedback(tabId, events, run = this._pageFeedbackRuns?.get(tabId)) {
+ return isPassivePageFeedback(events) || (!!events?.length && !!this._monitoringRetryCurrent(tabId, run)
+ && events.every(event => event.kind === 'dom' && ['page', 'unknown'].includes(event.source)
+ && event.frameId === 0 && !/^(?:base|style|link)(?:$|[\s#.])/.test(event.target || '')));
+ },
+
+ _monitoringRetryCurrent(tabId, run = this._pageFeedbackRuns?.get(tabId)) {
+ const retry = run?.monitoringRetry;
+ if (!retry || retry.invalidated || this._checkAbort(tabId) || this._hasPendingSteering?.(tabId)) return null;
+ const binding = retry.offer.binding, page = binding.monitoringPage;
+ const current = this._failureRecoveryBinding?.(tabId);
+ const matches = this.isRunning(tabId) && page && run.url === page.url
+ && run.frames.get(0)?.token === page.documentToken
+ && (!page.documentId || run.documents.get(0)?.id === page.documentId)
+ && this._steeringRuns?.get(tabId)?.requestId === retry.requestId
+ && ['conversationId', 'taskKey', 'taskSignature'].every(key => current?.[key] === binding[key]);
+ if (!matches) this._invalidateMonitoringRetry(tabId, 'binding_changed');
+ return matches ? retry : null;
+ },
+
+ _invalidateMonitoringRetry(tabId, reason) {
+ const run = this._pageFeedbackRuns?.get(tabId), retry = run?.monitoringRetry || this._monitoringRetryStarts?.get(tabId);
+ if (!retry || retry.invalidated) return;
+ retry.invalidated = true;
+ const runId = this.currentRunId?.get(tabId);
+ if (runId) trace.recordNote(runId, 0, 'monitoring_retry', { status: 'expired', reason });
+ },
+
+ async _beginPageFeedbackRun(tabId, kind, signal = null, monitoringRetry = null) {
this._pageFeedbackRuns ??= new Map();
const run = { token: token(), kind, events: new Map(), frames: new Map(), documents: new Map(),
revision: 0, lastUserAt: 0, lastActivityAt: 0, gestures: new Set(), gestureLeases: new Map(),
- navigation: null, url: '', onUpdate: null };
+ navigation: null, url: '', onUpdate: null, monitoringRetry };
this._pageFeedbackRuns.set(tabId, run);
const assertCurrent = () => {
if (signal) this._throwIfAborted(signal);
@@ -331,6 +361,7 @@ export const pageFeedbackMethods = {
pending: () => this._pageFeedbackBlocksCall(tabId, run.dispatchCall?.name, run.dispatchCall?.args),
allowPassiveRebase: () => run.dispatchCall?.allowPassiveRebase === true,
expectedModelSnapshot: () => run.dispatchCall?.dispatchState?.started ? undefined : run.dispatchCall?.expectedModelSnapshot,
+ relaxedMonitorGrantToken: () => this._monitoringRetryCurrent(tabId, run)?.grantToken,
clearNavigation: () => { run.navigation = null; },
frameForDocument: documentToken => [...run.frames].find(([, frame]) => frame.token === documentToken)?.[0],
dispatched: details => {
@@ -400,7 +431,9 @@ export const pageFeedbackMethods = {
run.frames.set(frameId, { token: documentToken, id: sender.documentId || '', seq: 0,
name: clean(frameName, 256) });
} else previous.name = clean(frameName, 256);
- return { active: true, runToken: run.token, documentToken };
+ const retry = frameId === 0 ? this._monitoringRetryCurrent(tabId, run) : null;
+ return { active: true, runToken: run.token, documentToken,
+ ...(retry ? { relaxedMonitorGrant: { grantToken: retry.grantToken, runToken: run.token, documentToken } } : {}) };
},
observePageFeedback(sender, feedback) {
@@ -479,6 +512,7 @@ export const pageFeedbackMethods = {
const run = this._pageFeedbackRuns?.get(details.tabId);
if (!run || !this.isRunning(details.tabId) || this._checkAbort(details.tabId)) return;
const frameId = Number(details.frameId) || 0;
+ if (frameId === 0) this._invalidateMonitoringRetry(details.tabId, 'navigation');
if (type === 'committed') {
run.frames.delete(frameId);
clearGestureLease(run, frameId);
@@ -526,6 +560,7 @@ export const pageFeedbackMethods = {
_queuePageFeedback(tabId, item) {
const run = this._pageFeedbackRuns?.get(tabId);
if (!run) return;
+ if (!this._isPassiveMonitoringFeedback(tabId, [item], run)) this._invalidateMonitoringRetry(tabId, 'intervention');
run.revision++;
const key = `${item.frameId}:${item.kind}:${item.target || ''}`;
const previous = run.events.get(key);
@@ -794,15 +829,18 @@ export const pageFeedbackMethods = {
// one content task. A snapshot taken after an older observation could bind
// a new recipient to a model decision that still saw the previous one.
const actionCaptureStarted = Date.now();
- let actionCaptureStatus = 'unavailable', actionTargetCount = 0;
+ let actionCaptureStatus = 'unavailable', actionTargetCount = 0, actionMonitoringRelaxed = false;
// Adapter proof is private and app-owned; unsupported pages return no
// preparation. Keep its owner bound across the same capture await.
const nativeChatOwner = this._nativeChatOwnerScope?.(tabId) || '';
+ const monitoringRetry = this._monitoringRetryCurrent(tabId, run);
+ if (monitoringRetry) monitoringRetry.started = true;
if (Array.isArray(messages) && /^https?:\/\//i.test(url) && state.frames.get(0)?.token) {
try {
const api = apiFor();
const capture = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_capture_model',
params: { runToken: run.token, includeTree: true,
+ ...(monitoringRetry ? { relaxedMonitorGrantToken: monitoringRetry.grantToken } : {}),
...(nativeChatOwner ? { nativeChatOwner } : {}),
...(run.uploadObservationScope?.documentToken === state.frames.get(0)?.token
&& run.uploadObservationScope.url === url ? { uploadTargets: run.uploadObservationScope.selectors } : {}),
@@ -820,9 +858,13 @@ export const pageFeedbackMethods = {
&& (!nativeChatOwner || nativeChatOwner === this._nativeChatOwnerScope?.(tabId))
&& this._pageFeedbackRuns?.get(tabId) === run && this._pageFeedbackStateCurrent(tabId, run, state, url)) {
actionCaptureStatus = 'ready';
+ actionMonitoringRelaxed = capture.monitoringRelaxed === true;
+ if (monitoringRetry && capture.monitoringRelaxed === false) this._invalidateMonitoringRetry(tabId, 'grant_expired');
actionTargetCount = Number.isSafeInteger(capture.targetCount) ? Math.max(0, Math.min(512, capture.targetCount)) : 0;
state.actionBinding = { snapshotToken: capture.snapshotToken, runToken: run.token, documentToken: capture.documentToken,
- focusedTargetAvailable: capture.focusedTargetAvailable === true };
+ focusedTargetAvailable: capture.focusedTargetAvailable === true,
+ ...(monitoringRetry && this._monitoringRetryCurrent(tabId, run) === monitoringRetry
+ ? { relaxedMonitorGrantToken: monitoringRetry.grantToken } : {}) };
if (capture.observationScopeCaptured === false) run.actionObservationScope = null;
run.actionCaptureTarget = null;
state.page = run.latestPage = capture.page;
@@ -844,6 +886,7 @@ export const pageFeedbackMethods = {
status: actionCaptureStatus, targetCount: actionTargetCount,
focusedTargetAvailable: state.actionBinding?.focusedTargetAvailable === true,
captureMs: Date.now() - actionCaptureStarted,
+ monitoringRelaxed: actionMonitoringRelaxed,
});
}
},
@@ -885,16 +928,35 @@ export const pageFeedbackMethods = {
const binding = state?.actionBinding;
if (run) { run.actionBindingUnavailable = null; run.actionContextUnverified = false; }
if (!run || !binding || run.modelState !== state || !this._pageFeedbackStateCurrent(tabId, run, state)
- || (run.events.size && !isPassivePageFeedback([...run.events.values()]))) return false;
+ || (run.events.size && !this._isPassiveMonitoringFeedback(tabId, [...run.events.values()], run))) return false;
try {
+ const retry = this._monitoringRetryCurrent(tabId, run);
+ const relaxedMonitorGrantToken = retry && retry.grantToken === binding.relaxedMonitorGrantToken ? retry.grantToken : null;
const response = await apiFor().tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_validate_model',
- params: { ...safeAction(name, args), runToken: run.token, snapshotToken: binding.snapshotToken } }, { frameId: 0 });
+ params: { ...safeAction(name, args), runToken: run.token, snapshotToken: binding.snapshotToken,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}) } }, { frameId: 0 });
const owned = response?.runToken === binding.runToken
&& response.documentToken === binding.documentToken && response.snapshotToken === binding.snapshotToken
&& this._pageFeedbackRuns?.get(tabId) === run && run.modelState === state
&& this._pageFeedbackStateCurrent(tabId, run, state)
- && (!run.events.size || isPassivePageFeedback([...run.events.values()]));
- const valid = owned && response.ready === true;
+ && (!run.events.size || this._isPassiveMonitoringFeedback(tabId, [...run.events.values()], run));
+ const valid = owned && response.ready === true
+ && (response.relaxed !== true || (relaxedMonitorGrantToken && this._monitoringRetryCurrent(tabId, run) === retry));
+ const certificationFailure = ['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit',
+ 'target_uncovered', 'context_uncertified'].includes(response?.certificationFailure) ? response.certificationFailure : null;
+ if (owned) {
+ const runId = this.currentRunId?.get(tabId);
+ if (runId) trace.recordNote(runId, 0, 'page_action_validation', {
+ status: valid ? 'ready' : 'blocked', monitoringRelaxed: valid && response.relaxed === true,
+ ...(certificationFailure ? { certificationFailure } : {}),
+ ...(response.uncertified === true ? { certificationFailure: certificationFailure || 'context_uncertified' } : {}),
+ ...(typeof response.reason === 'string' && ['target_changed', 'target_unresolved', 'target_uncovered',
+ 'focus_changed', 'intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired',
+ 'pending_feedback', 'target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed',
+ 'relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable'].includes(response.reason) ? { predicate: response.reason } : {}),
+ });
+ if (retry && !valid && response.uncertified !== true) this._invalidateMonitoringRetry(tabId, 'target_changed');
+ }
if (valid) {
run.validatedActionBindings ??= new Map();
run.validatedActionBindings.set(`${name}:${JSON.stringify(args)}`, binding.snapshotToken);
@@ -925,8 +987,10 @@ export const pageFeedbackMethods = {
|| !this._pageFeedbackStateCurrent(tabId, run, run.dispatchCall.modelState))) return true;
if (!run?.events.size) return false;
const events = [...run.events.values()];
- if (!isPassivePageFeedback(events) || !this._pageFeedbackStateCurrent(tabId, run, run.modelState)) return true;
- const policy = pageFeedbackCallPolicy(name, args, run.modelState, run.latestPage, events);
+ if (!this._isPassiveMonitoringFeedback(tabId, events, run) || !this._pageFeedbackStateCurrent(tabId, run, run.modelState)) return true;
+ const policyEvents = this._monitoringRetryCurrent(tabId, run)
+ ? events.map(event => event.source === 'unknown' && event.kind === 'dom' ? { ...event, source: 'page' } : event) : events;
+ const policy = pageFeedbackCallPolicy(name, args, run.modelState, run.latestPage, policyEvents);
// Targets need a fresh comparison first; the content monitor then checks
// their immutable node and action context at the actual dispatch boundary.
if (policy.kind === 'bound_target') return run.dispatchCall?.allowPassiveRebase !== true
@@ -960,8 +1024,8 @@ export const pageFeedbackMethods = {
this._flushPageFeedbackMessages(tabId, messages);
const recovery = run?.recoveryResult;
if (!recovery) return null;
- // Retry observes the current page through the normal gates. A context
- // failure never grants the publication-audit bypass reserved for that audit.
+ // A context retry relaxes monitoring through a private one-run grant;
+ // publication-audit recovery remains a separate authorization.
this._offerFailureRecovery?.(tabId, { stage: 'action_preparation', code: recovery.code,
message: recovery.message }, onUpdate);
onUpdate(this._effectiveRunMode?.(tabId, null) === 'ask' ? 'error' : 'warning', { message: recovery.message, code: recovery.code });
@@ -978,6 +1042,21 @@ export const pageFeedbackMethods = {
}
},
+ async _pageFeedbackPreparationFailure(tabId, name, args, state, diagnostic = {}) {
+ const run = this._pageFeedbackRuns?.get(tabId);
+ // Legacy preparation can hit passive churn after an uncertified original
+ // target was allowed with no queued feedback. Preserve the honest reason
+ // only if a new validation still proves that original target unchanged.
+ if (run?.actionContextUnverified && run.modelState === state) {
+ await this._validatePageFeedbackAction(tabId, name, args, state);
+ if (run.actionContextUnverified && this._pageFeedbackStateCurrent(tabId, run, state)) {
+ return { ...pageFeedbackPendingResult(), ...diagnostic, actionContextUnverified: true,
+ errorCode: 'action_context_unverified' };
+ }
+ }
+ return { ...pageFeedbackPendingResult(), ...diagnostic };
+ },
+
_accountPageFeedbackNoDispatch(tabId, name, result) {
if (result?.pageFeedbackPending !== true || result?.noDispatch !== true) return false;
const run = this._pageFeedbackRuns?.get(tabId);
@@ -998,7 +1077,7 @@ export const pageFeedbackMethods = {
const run = this._pageFeedbackRuns?.get(tabId);
if (!run?.events.size) return false;
if (await this._retainIndependentAskAnswer(tabId, [call], false, '', 'preparation')) return false;
- if (!isPassivePageFeedback([...run.events.values()])) {
+ if (!this._isPassiveMonitoringFeedback(tabId, [...run.events.values()], run)) {
if (askAnswerCandidate([call], false, '').trim() && automaticAnswerFeedback([...run.events.values()])
&& this._pageFeedbackAskOwnerCurrent(tabId, run, run.modelState)) {
const messages = [];
@@ -1072,7 +1151,9 @@ export const pageFeedbackMethods = {
&& this._pageFeedbackStateCurrent(tabId, run, state, url)));
const canRetain = (feedback, options) => readonlyResponse
? isPassivePageFeedback(feedback)
- : canRetainPageFeedbackCalls(responseToolCalls, state, page, feedback, options);
+ : canRetainPageFeedbackCalls(responseToolCalls, state, page,
+ this._monitoringRetryCurrent(tabId, run) ? feedback.map(event => event.source === 'unknown' && event.kind === 'dom'
+ ? { ...event, source: 'page' } : event) : feedback, options);
let unverifiedActionContext = false;
let retained = sameDocument && canRetain(events)
&& (!pending.length || canRetain(pending, { pending: true }));
@@ -1222,7 +1303,9 @@ export function installPageFeedback(Agent) {
const operationId = run && mutation ? token() : '';
let noDispatch = false;
const expectedModelSnapshot = modelBound ? modelState.actionBinding.snapshotToken : undefined;
- if (run) run.dispatchCall = { name, args, allowPassiveRebase: false, dispatchState, expectedModelSnapshot, modelState };
+ const relaxedMonitorGrantToken = expectedModelSnapshot && this._monitoringRetryCurrent(tabId, run)?.grantToken;
+ if (run) run.dispatchCall = { name, args, allowPassiveRebase: false, dispatchState, expectedModelSnapshot, modelState,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}) };
if (run && mutation && owner) {
owner.operationId = operationId;
owner.operationFrames.set(operationId, new Set([0]));
@@ -1231,6 +1314,7 @@ export function installPageFeedback(Agent) {
try {
prepared = await apiFor().tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_prepare',
params: { ...safeAction(name, args), runToken: run.token, operationId, allowPassiveRebase: allowPassiveRebase === true,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}),
...(expectedModelSnapshot ? { expectedModelSnapshot } : {}) } }, { frameId: 0 });
} catch {}
// Certified targets keep their pre-inference footprint across passive
@@ -1270,14 +1354,18 @@ export function installPageFeedback(Agent) {
}
}
if (run && mutation && this._hasPendingPageFeedback(tabId) && result?.noDispatch === true
- && !result.denied && !result.cancelled && !result.outcomeUnknown) return { ...result, pageFeedbackPending: true };
+ && !result.denied && !result.cancelled && !result.outcomeUnknown) {
+ const preparation = await this._pageFeedbackPreparationFailure(tabId, name, args, modelState);
+ return { ...result, pageFeedbackPending: true,
+ ...(preparation.actionContextUnverified ? { actionContextUnverified: true, errorCode: preparation.errorCode } : {}) };
+ }
return result;
} catch (error) {
noDispatch = !dispatchState.started;
if (error?.code === 'page_feedback_pending') {
const facts = preparationVetoDiagnostics(error.pagePreparationVeto);
const diagnostic = Object.keys(facts).length ? { preparationVeto: facts } : {};
- if (!dispatchState.started) return { ...pageFeedbackPendingResult(), ...diagnostic };
+ if (!dispatchState.started) return this._pageFeedbackPreparationFailure(tabId, name, args, modelState, diagnostic);
const interrupted = { success: false, dispatched: true, outcomeUnknown: true, retryable: false,
pageFeedbackPending: true, ...diagnostic, error: 'The browser changed after this action began. Inspect its outcome before attempting another mutation.' };
return this._finalizeToolResultOnce(tabId, name, args, interrupted, dispatchState);
diff --git a/src/chrome/src/background.js b/src/chrome/src/background.js
index 0c06e9152..b8ef7635e 100644
--- a/src/chrome/src/background.js
+++ b/src/chrome/src/background.js
@@ -1989,12 +1989,18 @@ function isPublicationAuditFailureUpdate(update) {
|| (update?.type === 'failure_recovery' && update?.data?.stage === 'publication_audit');
}
+function isActionPreparationFailureUpdate(update) {
+ return update?.type === 'failure_recovery' && update?.data?.stage === 'action_preparation'
+ && ['action_context_unverified', 'page_unstable'].includes(update?.data?.code);
+}
+
function runUpdatesSucceeded(updates = []) {
return !updates.some(update => (
update?.type === 'error' || update?.type === 'quota'
|| isClarificationRequiredRunUpdate(update)
|| isPersistenceDegradedRunUpdate(update)
|| isPublicationAuditFailureUpdate(update)
+ || isActionPreparationFailureUpdate(update)
|| isPlannerRequestFailureUpdate(update)
));
}
@@ -2008,6 +2014,7 @@ function terminalRunUiStatus(content, updates = [], error = null) {
if (updates.some(update => update?.type === 'error'
|| isPlannerRequestFailureUpdate(update) || isPersistenceDegradedRunUpdate(update)
|| isPublicationAuditFailureUpdate(update)
+ || isActionPreparationFailureUpdate(update)
|| update?.type === 'quota')) return 'failed';
if (updates.some(isClarificationRequiredRunUpdate)) return 'clarification_required';
return 'completed';
@@ -2031,6 +2038,7 @@ function askCompletionSucceededForBadge(result, updates = [], error = null) {
update?.type === 'error' || update?.type === 'quota'
|| isPersistenceDegradedRunUpdate(update)
|| isPublicationAuditFailureUpdate(update)
+ || isActionPreparationFailureUpdate(update)
|| update?.type === 'attachment_rejected'
|| update?.type === 'max_steps_reached'
|| update?.error
@@ -2282,6 +2290,7 @@ function launchDetachedRun(action, msg, sender) {
});
entry.promise = task;
task.catch((error) => {
+ agent._restoreFailureRecoveryStart?.(tabId, requestId);
rememberDetachedRunFailure(tabId, requestId, error);
console.warn(`[WebBrain] detached ${action} run failed:`, error);
}).finally(() => {
@@ -2962,9 +2971,14 @@ async function handleFailureRecoveryResponse(msg, sender) {
}
}
assertNoActiveTabRun(tabId);
- const result = agent.requestFailureRecovery(tabId, recoveryId, choice);
+ const result = agent.requestFailureRecovery(tabId, recoveryId, choice, msg.requestId);
if (!result?.ok || choice === 'give_up' || result.continue !== true) return result;
- return launchDetachedRun('continue', msg, sender);
+ try {
+ return launchDetachedRun('continue', msg, sender);
+ } catch (error) {
+ agent._restoreFailureRecoveryStart?.(tabId, msg.requestId);
+ throw error;
+ }
}
async function handleMessage(msg, sender) {
diff --git a/src/chrome/src/content/page-monitor.js b/src/chrome/src/content/page-monitor.js
index b63a8bd02..6f9f255df 100644
--- a/src/chrome/src/content/page-monitor.js
+++ b/src/chrome/src/content/page-monitor.js
@@ -42,6 +42,23 @@
let lastFeedbackDelivery = Promise.resolve();
let agentTurn = null, userTurn = null, lastUserTarget = null;
let identityPreparationCapture = null;
+ // Only the authenticated background state response can establish this grant.
+ // Model/page metadata and ordinary action parameters cannot create one.
+ let relaxedMonitorGrant = null;
+ const revokedRelaxedGrants = new Set();
+ const relaxedObservedBindings = new Set();
+ function revokeRelaxedGrant() {
+ if (relaxedMonitorGrant) revokedRelaxedGrants.add(relaxedMonitorGrant.grantToken);
+ relaxedMonitorGrant = null;
+ relaxedObservedBindings.clear();
+ }
+ function currentRelaxedGrant(token) {
+ return active && !!relaxedMonitorGrant && typeof token === 'string'
+ && token === relaxedMonitorGrant.grantToken && relaxedMonitorGrant.runToken === runToken
+ && relaxedMonitorGrant.documentToken === documentToken
+ && !revokedRelaxedGrants.has(token)
+ && relaxedMonitorGrant.url === location.href && relaxedMonitorGrant.baseURI === document.baseURI;
+ }
let matchedEvents = new WeakMap();
const activePointers = new Set();
const fenceAttribute = 'data-webbrain-page-revision';
@@ -120,6 +137,7 @@
};
const send = event => {
if (!active || disposed) return;
+ if (event.source !== 'agent' && (event.kind !== 'dom' || event.source === 'user')) revokeRelaxedGrant();
publishRevision();
const feedback = { ...event, runToken, documentToken, seq: ++seq, revision };
const pending = event.source !== 'agent';
@@ -157,8 +175,18 @@
|| proof.runToken !== runToken || proof.documentToken !== documentToken
|| proof.userAt !== lastUserAt || proof.interventionRevision !== interventionRevision
|| deepActiveElement() !== op.preparedBinding.target) continue;
- const before = proof.binding, after = actionFootprint(before.target, before.readonlyPreparation);
+ const before = proof.binding, after = before.relaxed
+ ? relaxedTargetBinding(before.target, before.grantToken) : actionFootprint(before.target, before.readonlyPreparation);
if (!after) continue;
+ if (before.relaxed) {
+ const original = proof.viewport;
+ if (!original || !sameRelaxedBinding(before, after)
+ || ![original, viewport].every(value => value.scale === 1 && value.visualWidth === value.width
+ && value.visualHeight === value.height && Math.abs(value.offsetX) < 1 && Math.abs(value.offsetY) < 1)
+ || ['width', 'height', 'visualWidth', 'visualHeight', 'scale', 'dpr'].some(key => original[key] !== viewport[key])) continue;
+ ownedPreparationBindings.set(op.preparedBinding, { ...proof, binding: after, viewportNotificationPending: false });
+ return true;
+ }
const originalContext = JSON.parse(before.context), currentContext = JSON.parse(after.context);
const original = originalContext[2];
if (![original, viewport].every(value => value.scale === 1 && value.visualWidth === value.width
@@ -205,6 +233,7 @@
const interact = (kind, el, { completeGesture = false, ...extra } = {}) => {
const isIgnored = ignored(el);
if (!active || (isIgnored && !completeGesture)) return;
+ revokeRelaxedGrant();
lastUserAt = Date.now(); lastUserTarget = isIgnored ? null : el; revision++;
const marker = {}; userTurn = marker;
setTimeout(() => { if (userTurn === marker) userTurn = null; }, 0);
@@ -371,9 +400,15 @@
const allowedEvent = data.kind === 'input'
? ['keydown', 'beforeinput', 'input', 'change', 'compositionstart', 'compositionend'].includes(event.type)
: data.kind === 'click' ? ['pointerover', 'mouseover', 'pointermove', 'pointerdown', 'mousedown', 'click'].includes(event.type) : false;
- const fresh = op && data.revision === op.preparedRevision && op.preparedRevision === revision
- && !domTimer && !unreported && lastUserAt <= op.userAt;
const signed = op && nativeMarkerSignature(op.nativeSecret, data) === data.signature;
+ const markerRevisionMatches = op && data.revision === op.preparedRevision;
+ const relaxedFresh = markerRevisionMatches && signed && markerTarget && nextSequence && allowedEvent
+ && data.documentToken === documentToken && data.runToken === runToken && op.preparedBinding?.relaxed === true
+ && (flushPendingMutations(op), sampleFormControls(op.target),
+ !nativeReceiverPredicate(op, { kind: data.kind }) && rebasePassivePreparation(op, { documentToken: data.documentToken }));
+ if (relaxedFresh) op.relaxedNativeRevision = data.revision;
+ const fresh = markerRevisionMatches && (op.preparedBinding?.relaxed ? relaxedFresh : (op.preparedRevision === revision
+ && !domTimer && !unreported && lastUserAt <= op.userAt));
if (data.documentToken === documentToken && data.runToken === runToken && op && markerTarget && nextSequence
&& fresh && signed && allowedEvent && ['input', 'click'].includes(data.kind)) {
// The capability is bound to the isolated-world prepared node and consumed once.
@@ -602,6 +637,7 @@
currentViewport(), window.scrollX, window.scrollY]) };
}
function samePreparedBinding(before, after) {
+ if (before?.relaxed || after?.relaxed) return sameRelaxedBinding(before, after);
if (!before || !after || before.target !== after.target || before.scope !== after.scope || before.hit !== after.hit || before.context !== after.context) return false;
return ['ancestors', 'nodes'].every(key => before[key].length === after[key].length
&& before[key].every((item, index) => item.node === after[key][index].node && item.value === after[key][index].value
@@ -616,6 +652,7 @@
? proof.binding : before;
}
function samePreparationSemantics(before, after) {
+ if (before?.relaxed) return sameRelaxedBinding(before, after);
if (!before?.model || !after?.model) return false;
if (![before, after].every(binding => binding.hit === binding.target || binding.target.contains?.(binding.hit))) return false;
const original = JSON.parse(before.context), current = JSON.parse(after.context);
@@ -1511,6 +1548,7 @@
...(readonlyPreparation ? { readonlyPreparation } : {}) };
}
function operationBinding(op, target = op?.preparedBinding?.target) {
+ if (op?.preparedBinding?.relaxed) return relaxedTargetBinding(target, op.preparedBinding.grantToken);
return op?.preparedBinding?.model === true ? actionFootprint(target, op.preparedBinding.readonlyPreparation) : preparedBinding(target);
}
function inputContinuationValue(item, target) {
@@ -1530,6 +1568,8 @@
}
function inputContinuationBindingMatches(op, after) {
const before = preparedBindingFor(op?.preparedBinding);
+ if (before?.relaxed) return op?.dispatched && op.kind === 'input'
+ && sameRelaxedBinding(before, after, true);
if (!op?.dispatched || op.kind !== 'input' || before?.model !== true || !after
|| before.target !== after.target || before.scope !== after.scope || before.hit !== after.hit || before.context !== after.context) return false;
return ['ancestors', 'nodes'].every(key => before[key].length === after[key].length
@@ -1543,7 +1583,11 @@
}
function operationBindingMatches(op, target = op?.preparedBinding?.target) {
const current = operationBinding(op, target);
- return samePreparedBinding(preparedBindingFor(op?.preparedBinding), current) || inputContinuationBindingMatches(op, current);
+ const matches = samePreparedBinding(preparedBindingFor(op?.preparedBinding), current) || inputContinuationBindingMatches(op, current);
+ if (op?.preparedBinding?.relaxed && (!matches || !relaxedActionability(target, { input: kindFor(op.modelBindingTool || op.tool) === 'input' }))) {
+ revokeRelaxedGrant(); return false;
+ }
+ return matches;
}
function modelTarget(params = {}) {
if (!modelActionTools.has(params.tool) || Number.isFinite(params.x) || Number.isFinite(params.y)
@@ -1573,20 +1617,125 @@
value]);
return { value: identity, parent: bindingParent(target), form: target.form || target.closest('form,[role="form"]') };
}
+ function relaxedTargetBinding(target, grantToken) {
+ if (!currentRelaxedGrant(grantToken) || !(target instanceof Element) || !target.isConnected
+ || target.ownerDocument !== document || !modelCandidateIdentity(target)) return null;
+ const ancestors = [];
+ for (let node = bindingParent(target); node; node = bindingParent(node)) {
+ if (ancestors.length >= 64) return null;
+ ancestors.push({ node, value: '' });
+ }
+ // Focus/scroll decoration and extension transport markers do not change
+ // the control's meaning. All other authored attributes remain exact.
+ const attributes = [...target.attributes].filter(attribute => !['class', 'style',
+ 'data-webbrain-native-target', 'data-webbrain-native-action'].includes(attribute.name))
+ .map(attribute => [attribute.name, attribute.value]).sort(([a], [b]) => a.localeCompare(b));
+ const nativeControl = /^(INPUT|TEXTAREA|SELECT|OPTION)$/.test(target.tagName);
+ const fixed = [target.tagName, attributes, target.disabled === true, target.readOnly === true,
+ target.isContentEditable === true, target.type || null];
+ const content = [nativeControl ? target.value : null, target.textContent,
+ nativeControl ? [target.checked === true, target.selected === true, target.selectedIndex ?? null] : null];
+ const form = target.form || target.closest('form,[role="form"]');
+ if (form && (form.attributes.length > 64 || [...form.attributes].some(attribute => attribute.value.length > 8192))) return null;
+ const formIdentity = form ? JSON.stringify([[...form.attributes].filter(attribute => !['class', 'style'].includes(attribute.name))
+ .map(attribute => [attribute.name, attribute.value]).sort(([a], [b]) => a.localeCompare(b)),
+ form.action || null, form.method || null, form.target || null, form.enctype || null]) : null;
+ return { model: true, relaxed: true, grantToken, target, scope: bindingParent(target), form, formIdentity,
+ clickHandler: target.onclick, submitHandler: form?.onsubmit,
+ identity: JSON.stringify([fixed, content]),
+ continuationIdentity: JSON.stringify([fixed, target.isContentEditable ? null : target.textContent]),
+ ancestors, nodes: [], hit: target, context: JSON.stringify([location.href, document.baseURI]) };
+ }
+ function sameRelaxedBinding(before, after, continuingInput = false) {
+ return !!before?.relaxed && !!after?.relaxed && currentRelaxedGrant(before.grantToken)
+ && before.grantToken === after.grantToken && before.target === after.target && before.scope === after.scope
+ && before.form === after.form && before.formIdentity === after.formIdentity && before.context === after.context
+ && before.clickHandler === after.clickHandler && before.submitHandler === after.submitHandler
+ && (continuingInput ? before.continuationIdentity === after.continuationIdentity : before.identity === after.identity)
+ && before.ancestors.length === after.ancestors.length
+ && before.ancestors.every((item, index) => item.node === after.ancestors[index].node);
+ }
+ function sealRelaxedOwnedInput(op) {
+ const original = op?.preparedBinding;
+ if (!original?.relaxed || !op.dispatched || op.kind !== 'input' || op.typingPreparationOnly
+ || !currentRelaxedGrant(original.grantToken) || lastUserAt !== op.userAt) return;
+ const current = relaxedTargetBinding(original.target, original.grantToken);
+ if (!sameRelaxedBinding(original, current, true)) { revokeRelaxedGrant(); return; }
+ // Preserve the immutable inference/dispatch certificate. Only the local
+ // drift baseline advances to this exact owned input's completed body;
+ // otherwise its delayed observer checkpoint would expire the next action.
+ for (const binding of relaxedObservedBindings) if (binding.target === original.target) relaxedObservedBindings.delete(binding);
+ relaxedObservedBindings.add(current);
+ }
+ function finishRelaxedOwnedAction(op) {
+ sealRelaxedOwnedInput(op);
+ if (!op?.preparedBinding?.relaxed || !op.dispatched || op.kind !== 'click'
+ || op.pointerPreparationOnly || op.selectorPreparationOnly || !currentRelaxedGrant(op.preparedBinding.grantToken)
+ || lastUserAt !== op.userAt) return;
+ // A completed click can open its menu or remove its own control. The
+ // old snapshot stays immutable; the next action needs a fresh capture.
+ for (const binding of relaxedObservedBindings) if (binding.target === op.preparedBinding.target) relaxedObservedBindings.delete(binding);
+ }
+ function relaxedSemanticMutation(binding, record) {
+ if (record.type === 'attributes') return [binding.target, binding.form].includes(record.target)
+ && !['class', 'style', 'data-webbrain-native-target', 'data-webbrain-native-action'].includes(record.attributeName)
+ && record.oldValue !== record.target.getAttribute(record.attributeName);
+ if (record.type === 'childList' && [...record.removedNodes].some(node => node === binding.target
+ || node.contains?.(binding.target) || binding.ancestors.some(item => item.node === node))) return true;
+ const node = record.target.nodeType === 1 ? record.target : record.target.parentElement;
+ return ['characterData', 'childList'].includes(record.type)
+ && (node === binding.target || binding.target.contains(node));
+ }
+ function relaxedActionability(target, { hit = false, input = false } = {}) {
+ if (!(target instanceof Element) || !target.isConnected || target.ownerDocument !== document || !visible(target)
+ || !target.matches(modelControlSelector) || target.disabled === true || (input && target.readOnly === true)) return false;
+ for (let node = target; node; node = bindingParent(node)) {
+ const style = getComputedStyle(node);
+ if (node.inert || node.getAttribute('aria-hidden') === 'true' || node.getAttribute('aria-disabled') === 'true'
+ || node.matches('fieldset[disabled]') || style.display === 'none' || ['hidden', 'collapse'].includes(style.visibility)
+ || style.contentVisibility === 'hidden' || Number(style.opacity) === 0) return false;
+ }
+ const rect = target.getBoundingClientRect();
+ if (!(rect.width > 0 && rect.height > 0)) return false;
+ if (!hit) return true;
+ const receiver = deepElementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2);
+ return !!receiver && (receiver === target || target.contains(receiver));
+ }
function modelBindingResult(params = {}) {
const snapshot = modelSnapshots.get(params.snapshotToken || params.expectedModelSnapshot);
if (!active || params.runToken !== runToken || snapshot?.runToken !== runToken) return { reason: 'model_snapshot_invalid' };
if (!snapshot || snapshot.until < Date.now()) return { reason: 'model_snapshot_expired' };
- if (snapshot.interventionRevision !== interventionRevision || snapshot.userAt !== lastUserAt || interventionUnreported)
+ const relaxed = snapshot.relaxedGrantToken && snapshot.relaxedGrantToken === params.relaxedMonitorGrantToken
+ && currentRelaxedGrant(params.relaxedMonitorGrantToken);
+ if (snapshot.relaxedGrantToken && !relaxed) return { reason: 'relaxed_grant_expired' };
+ if (snapshot.userAt !== lastUserAt || (!relaxed && (snapshot.interventionRevision !== interventionRevision || interventionUnreported)))
return { reason: 'intervention_changed' };
- if (pendingDOM && pendingDOM.source !== 'page') return { reason: 'pending_feedback' };
+ if (!relaxed && pendingDOM && pendingDOM.source !== 'page') return { reason: 'pending_feedback' };
const target = modelTarget(params);
- if (!target) return params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason: 'target_unresolved' };
+ if (!target) {
+ if (relaxed) { revokeRelaxedGrant(); return { reason: 'target_unresolved' }; }
+ return params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason: 'target_unresolved' };
+ }
const before = snapshot.targets.get(target), previous = snapshot.uncertified.get(target);
- if (!before && !previous) return { reason: 'target_uncovered' };
- if (params.ref_id && snapshot.hasRefs && snapshot.refs.get(params.ref_id) !== target) return { reason: 'target_changed' };
+ if (!relaxed && !before && !previous) return { reason: 'target_uncovered', certificationFailure: snapshot.captureFailures.get(target) || 'identity_unobserved' };
+ if (params.ref_id && snapshot.hasRefs && snapshot.refs.get(params.ref_id) !== target) {
+ if (relaxed) revokeRelaxedGrant();
+ return { reason: 'target_changed' };
+ }
if (!params.ref_id && !params.selector && !params.nativeTarget && !params.textMatch && modelFocusTools.has(params.tool)
&& snapshot.focusTarget !== target) return { reason: 'focus_changed' };
+ if (relaxed) {
+ const original = snapshot.relaxedTargets.get(target);
+ const current = relaxedTargetBinding(target, params.relaxedMonitorGrantToken);
+ if (!original) return { reason: 'target_uncovered', certificationFailure: snapshot.captureFailures.get(target) || 'identity_unobserved' };
+ if (snapshot.relaxedInvalidated.has(target) || !sameRelaxedBinding(original, current)) { revokeRelaxedGrant(); return { reason: 'target_changed' }; }
+ if (!relaxedActionability(target, { input: kindFor(params.tool) === 'input' })) {
+ revokeRelaxedGrant(); return { reason: 'target_not_actionable' };
+ }
+ for (const binding of relaxedObservedBindings) if (binding.target === target) relaxedObservedBindings.delete(binding);
+ relaxedObservedBindings.add(original);
+ return { binding: original, relaxed: true, certificationFailure: snapshot.captureFailures.get(target) || undefined };
+ }
if (before?.readonlyPreparation && params.tool !== 'click_ax') return {};
if (before) {
const reason = bindingPredicate(preparedBindingFor(before), actionFootprint(target, before.readonlyPreparation));
@@ -1594,7 +1743,7 @@
}
const current = modelCandidateIdentity(target);
return current && previous.value === current.value && previous.parent === current.parent && previous.form === current.form
- ? { uncertified: true } : {};
+ ? { uncertified: true, certificationFailure: snapshot.captureFailures.get(target) || 'owner_unresolved' } : {};
}
function captureModel(params = {}) {
if (!active || params.runToken !== runToken) return { ready: false, runToken, documentToken };
@@ -1700,6 +1849,8 @@
} catch { return { ready: false, runToken, documentToken, reason: 'observation_unavailable' }; }
}
const targets = new Map(), uncertified = new WeakMap(), refs = new Map(), seen = new Set(), publicNodes = new Set();
+ const relaxedGrantToken = page && currentRelaxedGrant(params.relaxedMonitorGrantToken) ? params.relaxedMonitorGrantToken : null;
+ const relaxedTargets = new Map(), relaxedInvalidated = new WeakSet(), captureFailures = new WeakMap();
if (page) for (const { ref, end } of observedRefs) {
if (end > page.pageContent.length) continue;
const node = window.__wb_ax_lookup?.(ref);
@@ -1844,13 +1995,20 @@
// inputs still need the existing structural owner proof.
if (page && !publicNodes.has(node) && !(node.tagName === 'INPUT' && node.type === 'file')) return;
seen.add(node);
- const footprint = targets.size < MODEL_TARGET_LIMIT ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;
- if (footprint && !observedFootprint(footprint)) return;
+ const identity = modelCandidateIdentity(node);
+ if (!identity) captureFailures.set(node, 'capture_limit');
+ if (relaxedGrantToken && publicNodes.has(node) && node.matches(modelControlSelector)) {
+ const binding = relaxedTargetBinding(node, relaxedGrantToken);
+ if (binding) relaxedTargets.set(node, binding);
+ }
+ const limitReached = targets.size >= MODEL_TARGET_LIMIT;
+ const footprint = !limitReached ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;
+ if (footprint && !observedFootprint(footprint)) { captureFailures.set(node, 'identity_unobserved'); return; }
if (!footprint && page && !publicNodes.has(node)) return;
if (footprint) targets.set(node, footprint);
else {
- const identity = modelCandidateIdentity(node);
if (identity) uncertified.set(node, identity);
+ if (!captureFailures.has(node)) captureFailures.set(node, limitReached ? 'capture_limit' : 'owner_unresolved');
}
};
// Recovery observes this target's current semantic owner in the same task
@@ -1888,10 +2046,13 @@
// Metadata-only tests/integration use an existing AX lookup without a new
// tree. Record those refs lazily only when they resolve to an original node.
modelSnapshots.set(snapshotToken, { runToken, until: Date.now() + MODEL_SNAPSHOT_MS, targets, uncertified, refs,
+ relaxedGrantToken, relaxedTargets, relaxedInvalidated, captureFailures,
hasRefs: !!page || !!window.__wbElementMap, focusTarget, userAt: lastUserAt, interventionRevision });
for (const [key, snapshot] of modelSnapshots) if (snapshot.until < Date.now()) modelSnapshots.delete(key);
while (modelSnapshots.size > 2) modelSnapshots.delete(modelSnapshots.keys().next().value);
return { ready: true, runToken, documentToken, snapshotToken, targetCount: targets.size, focusedTargetAvailable,
+ monitoringRelaxed: !!relaxedGrantToken && currentRelaxedGrant(relaxedGrantToken),
+ ...(priorityTarget && captureFailures.has(priorityTarget) ? { certificationFailure: captureFailures.get(priorityTarget) } : {}),
...(params.observationRef ? { observationScopeCaptured } : {}),
...(page ? { page } : {}) };
}
@@ -1903,6 +2064,15 @@
&& !operationBindingMatches(op);
}
function nativeReceiverPredicate(op, params) {
+ if (op?.preparedBinding?.relaxed && (params.ref_id || params.selector || params.textMatch)
+ && resolveTarget(params) !== op.preparedBinding.target) {
+ revokeRelaxedGrant(); return 'target_changed';
+ }
+ if (op?.preparedBinding?.relaxed && ['click', 'input'].includes(params.kind)
+ && params.pointerPreparationOnly !== true
+ && !relaxedActionability(op.preparedBinding.target, { hit: true, input: params.kind === 'input' })) {
+ revokeRelaxedGrant(); return 'target_not_actionable';
+ }
if (!params.nativeTarget || op?.preparedBinding?.model !== true) return '';
// Resolve the actual transport marker alone; a stale marker cannot fall
// back to the requested selector or ref and borrow its certificate.
@@ -1920,12 +2090,13 @@
return 'target_changed';
}
function rebasePassivePreparation(op, params = {}) {
+ const relaxed = op?.preparedBinding?.relaxed === true && currentRelaxedGrant(op.preparedBinding.grantToken);
const continuingInput = op?.dispatched && op.kind === 'input' && op.preparedBinding?.model === true;
const continuingPreparation = op?.dispatched && (op.pointerPreparationOnly === true || op.typingPreparationOnly === true) && op.preparedBinding?.model === true;
if (!active || !op || op.allowPassiveRebase !== true || (op.dispatched && !continuingInput && !continuingPreparation) || op.coordinateSensitive || op.layoutInvalidated
- || op.until < Date.now() || lastUserAt > op.userAt || interventionUnreported
- || op.preparedInterventionRevision !== interventionRevision
- || (pendingDOM && pendingDOM.source !== 'page')
+ || op.until < Date.now() || lastUserAt > op.userAt || (!relaxed && interventionUnreported)
+ || (!relaxed && op.preparedInterventionRevision !== interventionRevision)
+ || (!relaxed && pendingDOM && pendingDOM.source !== 'page')
|| (params.documentToken && params.documentToken !== documentToken)) return false;
const independentNavigation = op.tool === 'navigate';
if (independentNavigation) {
@@ -2199,6 +2370,32 @@
}
function onMutations(records) {
if (!active) return;
+ // Every captured candidate retains its own tombstone, even before the
+ // model selects it. Churn on an unselected control does not expire the
+ // run; selecting a target changed during inference does.
+ if (relaxedMonitorGrant) for (const snapshot of modelSnapshots.values()) {
+ if (!snapshot.relaxedGrantToken || !currentRelaxedGrant(snapshot.relaxedGrantToken)) continue;
+ for (const binding of snapshot.relaxedTargets.values()) if (!snapshot.relaxedInvalidated.has(binding.target)
+ && (records.some(record => relaxedSemanticMutation(binding, record))
+ || (records.some(record => record.type === 'control' && record.target === binding.target)
+ && !sameRelaxedBinding(binding, relaxedTargetBinding(binding.target, binding.grantToken))))) snapshot.relaxedInvalidated.add(binding.target);
+ }
+ // Mutation history also fences an observed node that was changed and then
+ // restored before dispatch. Unrelated live-page churn does not spend the
+ // grant; input continuation can edit only its original editable contents.
+ if (relaxedMonitorGrant) for (const binding of relaxedObservedBindings) {
+ const owned = [...operations.values()].find(op => op.preparedBinding?.relaxed && op.preparedBinding.target === binding.target
+ && op.dispatched && !op.finished && op.kind === 'input' && lastUserAt <= op.userAt);
+ const ownedClick = agentTurn && lastUserAt <= agentTurn.userAt && [...operations.values()].some(op => op.preparedBinding?.relaxed
+ && op.preparedBinding.target === binding.target && op.dispatched && !op.finished && op.kind === 'click'
+ && !op.pointerPreparationOnly && !op.selectorPreparationOnly && op.userAt === agentTurn.userAt);
+ if (ownedClick) continue;
+ const material = records.some(record => relaxedSemanticMutation(binding, record)
+ && !(owned && ['characterData', 'childList'].includes(record.type)));
+ if (material || !sameRelaxedBinding(binding, relaxedTargetBinding(binding.target, binding.grantToken), !!owned)) {
+ revokeRelaxedGrant(); break;
+ }
+ }
let changed = false, attributed = false, source = 'page', target = '';
let diagnosticRecord = null;
const diagnostic = el => {
@@ -2387,9 +2584,20 @@
}
function start(state) {
if (disposed || !state?.active || state.documentToken !== documentToken) return;
- if (active && runToken === state.runToken) return;
+ const grant = state.relaxedMonitorGrant;
+ const authorizedGrant = grant && typeof grant.grantToken === 'string' && grant.grantToken.length >= 16
+ && grant.grantToken.length <= 256 && grant.runToken === state.runToken && grant.documentToken === documentToken
+ && !revokedRelaxedGrants.has(grant.grantToken)
+ ? Object.freeze({ ...grant, url: location.href, baseURI: document.baseURI }) : null;
+ if (active && runToken === state.runToken) {
+ if (relaxedMonitorGrant && (relaxedMonitorGrant.grantToken !== authorizedGrant?.grantToken
+ || !currentRelaxedGrant(relaxedMonitorGrant.grantToken))) revokeRelaxedGrant();
+ relaxedMonitorGrant = authorizedGrant && !revokedRelaxedGrants.has(authorizedGrant.grantToken) ? authorizedGrant : null;
+ return;
+ }
stop();
active = true; runToken = state.runToken; seq = 0; revision = 0; interventionRevision = 0; interventionDiagnostic = ''; publishRevision();
+ relaxedMonitorGrant = authorizedGrant;
observer = new MutationObserver(onMutations);
observeRoot(document);
controlTimer = setInterval(() => { sampleFormControls(); samplePreparedTargets(); }, 250);
@@ -2415,10 +2623,16 @@
event.stopImmediatePropagation();
flushPendingMutations(op);
sampleFormControls(op?.target);
- if (!active || guard.runToken !== runToken || guard.documentToken !== documentToken || guard.revision !== revision
+ const relaxed = op?.preparedBinding?.relaxed === true && guard.revision === op.preparedRevision
+ && guard.runToken === runToken && guard.documentToken === documentToken
+ && !phaseAlreadyConsumed && op.kind === phaseKind
+ && !nativeReceiverPredicate(op, { kind: phaseKind }) && rebasePassivePreparation(op, { documentToken: guard.documentToken });
+ if (relaxed) op.relaxedNativeRevision = guard.revision;
+ if (!active || guard.runToken !== runToken || guard.documentToken !== documentToken || (!relaxed && guard.revision !== revision)
+ || (op?.preparedBinding?.relaxed && !relaxed)
|| !op || phaseAlreadyConsumed || op.kind !== phaseKind
|| (phase === 'input' && elementFor(event) !== op.target && elementFor(event) !== op.focusTarget)
- || op.preparedRevision !== revision || domTimer || unreported || lastUserAt > op.userAt) {
+ || op.preparedRevision !== revision || (!relaxed && (domTimer || unreported)) || lastUserAt > op.userAt) {
event.preventDefault(); return;
}
if (actionPhase) {
@@ -2641,6 +2855,7 @@
listen(window, 'pagehide', () => stop());
}
function stop() {
+ revokeRelaxedGrant();
active = false; runToken = ''; activePointers.clear(); pointerHeld = false; composing = false;
clearTimeout(agentResize?.timer); agentResize = null;
lastUserAt = 0; lastViewport = ''; userTurn = null; lastUserTarget = null; matchedEvents = new WeakMap(); controlIterator = null; preparedOperationCursor = 0;
@@ -2692,8 +2907,9 @@
const bindingResult = expected ? modelBindingResult(params) : null;
const certified = expected ? bindingResult.binding : inherited
&& previous.runToken === runToken && previous.until >= Date.now()
- && previous.preparedInterventionRevision === interventionRevision && lastUserAt === previous.userAt
- && !interventionUnreported && (!pendingDOM || pendingDOM.source === 'page')
+ && lastUserAt === previous.userAt
+ && (previous.preparedBinding.relaxed ? currentRelaxedGrant(previous.preparedBinding.grantToken)
+ : previous.preparedInterventionRevision === interventionRevision && !interventionUnreported && (!pendingDOM || pendingDOM.source === 'page'))
&& !previous.modelBindingInvalid && !previous.layoutInvalidated
&& operationTarget === previous.preparedBinding.target
&& operationBindingMatches(previous, operationTarget) ? previous.preparedBinding : null;
@@ -2717,7 +2933,10 @@
coordinatePoint: coordinateSensitive ? { x: params.x, y: params.y } : null,
coordinateRect: rectFor(coordinateTarget),
until: Date.now() + 30000, dispatched: false, pointerPreparationOnly: false, selectorPreparationOnly: false, consequentialInputPending: false, userAt: lastUserAt, preparedRevision: revision });
- return proofRequired ? { modelBindingValid: !!certified } : {};
+ return proofRequired ? { modelBindingValid: !!certified,
+ ...(certified?.relaxed ? { relaxed: true, bindingKind: 'relaxed_target' } : {}),
+ ...(bindingResult?.reason ? { reason: bindingResult.reason } : {}),
+ ...(bindingResult?.certificationFailure ? { certificationFailure: bindingResult.certificationFailure } : {}) } : {};
}
// The second argument is private producer provenance, never message/model params.
function dispatch(params, selectorPreparationOnly = false, typingPreparationOnly = false) {
@@ -2773,7 +2992,9 @@
const layoutChanged = coordinatePreparationShifted(op);
const bindingChanged = preparedBindingChanged(op);
const rebased = !layoutChanged && rebasePassivePreparation(op, params);
+ const receiverPredicate = nativeReceiverPredicate(op, params);
if (!active || !op || op.modelBindingInvalid || layoutChanged || bindingChanged || (!rebased && (domTimer || unreported))
+ || receiverPredicate
|| (params.element && op.target !== params.element)
|| lastUserAt > op.userAt
|| (Number.isFinite(op.preparedRevision) && op.preparedRevision !== revision)) {
@@ -2794,6 +3015,9 @@
const currentFocus = deepActiveElement();
const rebindingFocus = params.rebindFocus === true;
const fileAssignment = op?.target?.tagName === 'INPUT' && op.target.type === 'file';
+ if (op?.preparedBinding?.relaxed && !relaxedActionability(op.preparedBinding.target, {
+ hit: ['click', 'input'].includes(op.kind), input: op.kind === 'input',
+ })) { revokeRelaxedGrant(); return false; }
// A companion validates once before writing its marker and again before
// the native key. Revalidating the same consumed Tab rebind is safe only
// while its native sequence and exact deeply focused node remain current.
@@ -2805,7 +3029,9 @@
const shifted = coordinatePreparationShifted(op);
const bindingChanged = preparedBindingChanged(op, { allowFocusRebind: focusRebindAvailable });
const preparedRevision = op?.preparedRevision;
- const suppliedRevisionMatches = Number(params.revision) === preparedRevision;
+ const suppliedRevisionMatches = Number(params.revision) === preparedRevision
+ || (op?.preparedBinding?.relaxed && currentRelaxedGrant(op.preparedBinding.grantToken)
+ && Number(params.revision) === op.relaxedNativeRevision);
const rebased = suppliedRevisionMatches && !shifted && rebasePassivePreparation(op, params);
if (!active || params.runToken !== runToken || params.documentToken !== documentToken || !op || op.modelBindingInvalid || shifted || bindingChanged
|| !suppliedRevisionMatches || op.preparedRevision !== revision
@@ -2845,15 +3071,20 @@
const previous = localOperation;
const target = resolveTarget({ ...params, textMatch: action === 'click' ? params.text : undefined })
|| (kindFor(action) === 'input' ? deepActiveElement() : null);
+ const bindingTool = action === 'ax_prepare_field_for_trusted_type' && params.expectedModelSnapshot
+ && currentRelaxedGrant(params.relaxedMonitorGrantToken) ? 'set_field' : action;
const approved = [...operations.values()].reverse().find(op => op.allowPassiveRebase === true && !op.dispatched
&& op.preparedBinding?.target === target && op.until >= Date.now()
- && op.preparedInterventionRevision === interventionRevision && lastUserAt <= op.userAt && !interventionUnreported
- && (!pendingDOM || pendingDOM.source === 'page')
+ && lastUserAt <= op.userAt
+ && (op.preparedBinding.relaxed ? currentRelaxedGrant(op.preparedBinding.grantToken)
+ : op.preparedInterventionRevision === interventionRevision && !interventionUnreported && (!pendingDOM || pendingDOM.source === 'page'))
&& operationBindingMatches(op, target));
prepare({ selector: params.selector, ref_id: params.ref_id, x: params.x, y: params.y,
- textMatch: action === 'click' ? params.text : undefined, tool: action, operationId, runToken,
+ textMatch: action === 'click' ? params.text : undefined, tool: bindingTool, operationId, runToken,
...(params.expectedModelSnapshot !== undefined || approved?.expectedModelSnapshot !== undefined
? { expectedModelSnapshot: params.expectedModelSnapshot ?? approved.expectedModelSnapshot } : {}),
+ ...(approved?.preparedBinding?.relaxed ? { relaxedMonitorGrantToken: approved.preparedBinding.grantToken } : {}),
+ ...(bindingTool !== action ? { relaxedMonitorGrantToken: params.relaxedMonitorGrantToken } : {}),
allowPassiveRebase: params.allowPassiveRebase === true || !!approved });
if (approved?.preparedBinding?.model === true) {
const local = operations.get(operationId);
@@ -2873,7 +3104,7 @@
if (!params._bidiPrepare) beforeLocalDispatch({ preparation: true });
return () => {
const op = operations.get(operationId);
- if (op) { op.until = Date.now() + 200; op.finished = true; }
+ if (op) { finishRelaxedOwnedAction(op); op.until = Date.now() + 200; op.finished = true; }
localOperation = previous;
};
}
@@ -2885,8 +3116,9 @@
const layoutChanged = coordinatePreparationShifted(operation);
const bindingChanged = preparedBindingChanged(operation);
const rebased = !layoutChanged && rebasePassivePreparation(operation, { element: target });
+ const receiverPredicate = !preparation ? nativeReceiverPredicate(operation, { kind: kind || localOperation.kind }) : '';
if (rebased) localOperation.revision = operation.preparedRevision;
- if (operation?.modelBindingInvalid || layoutChanged || bindingChanged || (!rebased && (domTimer || unreported)) || lastUserAt > localOperation.userAt || revision !== localOperation.revision) {
+ if (operation?.modelBindingInvalid || layoutChanged || bindingChanged || receiverPredicate || (!rebased && (domTimer || unreported)) || lastUserAt > localOperation.userAt || revision !== localOperation.revision) {
const error = new Error('Browser changed during action preparation. Re-observe before acting.');
error.code = 'page_feedback_pending'; error.dispatched = localOperation.started === true;
error.pagePreparationVeto = preparationVetoFacts(operation, { layoutChanged, bindingChanged, target });
@@ -2918,7 +3150,7 @@
beforeLocalDispatch({ preparation: true, kind, target });
const before = preparedBindingFor(op.preparedBinding);
const previousProof = ownedPreparationBindings.get(op.preparedBinding);
- const previousScrollX = window.scrollX, previousScrollY = window.scrollY;
+ const previousScrollX = window.scrollX, previousScrollY = window.scrollY, preparationViewport = currentViewport();
const preparedIntervention = interventionRevision, preparedUserAt = lastUserAt, preparedRun = runToken;
dispatch({ ...localOperation, runToken, kind, element: target, scrollIntoView: kind === 'scroll' || op.windowScroll,
addScrollTarget: true,
@@ -2928,9 +3160,9 @@
try {
const result = callback();
flushPendingMutations(op); sampleFormControls(target);
- const after = actionFootprint(target, before.readonlyPreparation);
+ const after = before.relaxed ? relaxedTargetBinding(target, before.grantToken) : actionFootprint(target, before.readonlyPreparation);
const changedAuthority = !active || runToken !== preparedRun || lastUserAt !== preparedUserAt
- || interventionRevision !== preparedIntervention || interventionUnreported;
+ || (before.relaxed ? !currentRelaxedGrant(before.grantToken) : interventionRevision !== preparedIntervention || interventionUnreported);
const changedFocus = kind === 'focus' && deepActiveElement() !== target;
if (changedAuthority || changedFocus || !samePreparationSemantics(before, after)) {
const error = new Error('Browser changed during action preparation. Re-observe before acting.');
@@ -2942,6 +3174,7 @@
// derived geometry proof for this original node is shared with its owner.
ownedPreparationBindings.set(op.preparedBinding, { binding: after, runToken, documentToken,
userAt: lastUserAt, interventionRevision,
+ ...(before.relaxed ? { viewport: preparationViewport } : {}),
scrollPreparationUntil: kind === 'scroll' && (previousScrollX !== window.scrollX || previousScrollY !== window.scrollY)
? Date.now() + 200 : previousProof?.scrollPreparationUntil || 0,
viewportNotificationPending: kind === 'scroll' && (previousScrollX !== window.scrollX || previousScrollY !== window.scrollY)
@@ -3020,6 +3253,8 @@
flushPendingMutations(); sampleFormControls();
const result = modelBindingResult(params);
respond({ ready: !!result.binding, ...(result.uncertified ? { uncertified: true } : {}),
+ ...(result.relaxed ? { relaxed: true, bindingKind: 'relaxed_target' } : {}),
+ ...(result.certificationFailure ? { certificationFailure: result.certificationFailure } : {}),
...(result.reason ? { reason: result.reason } : {}), runToken, documentToken, snapshotToken: params.snapshotToken });
}
else if (msg.action === 'page_monitor_prepare') { respond({ ready: true, ...prepare(msg.params || {}) }); }
@@ -3077,7 +3312,7 @@
&& (!active || params.runToken !== runToken || params.documentToken !== documentToken || !op)) {
respond({ ready: false }); return;
}
- if (op) { op.until = Date.now() + 200; op.finished = true; }
+ if (op) { finishRelaxedOwnedAction(op); op.until = Date.now() + 200; op.finished = true; }
respond({ ready: true });
}
};
diff --git a/src/chrome/src/trace/privacy.js b/src/chrome/src/trace/privacy.js
index 9eb0b0574..816d70045 100644
--- a/src/chrome/src/trace/privacy.js
+++ b/src/chrome/src/trace/privacy.js
@@ -40,6 +40,7 @@ const NOTE_METADATA_FIELDS = [
'success', 'approved', 'outcome', 'verification', 'count',
'adapter', 'revision', 'notesInjected', 'workflowSchema', 'job', 'template',
'validationFailure', 'candidateAttempt', 'stage', 'retryMode',
+ 'certificationFailure', 'monitoringRelaxed', 'predicate', 'targetCount', 'focusedTargetAvailable', 'captureMs',
];
// Checker diagnostics are finite protocol labels, never arbitrary model text
@@ -73,7 +74,13 @@ const RECOVERY_STAGES = new Set([
'completion_verification', 'verification', 'action_preparation',
'response_generation', 'browser_interaction',
]);
-const RECOVERY_RETRY_MODES = new Set(['resume', 'publication_without_audit', 'publication_audit_bypass']);
+const RECOVERY_RETRY_MODES = new Set(['resume', 'publication_without_audit', 'publication_audit_bypass', 'action_monitoring_relaxed']);
+const CERTIFICATION_FAILURES = new Set(['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit',
+ 'target_uncovered', 'context_uncertified']);
+const ACTION_PREDICATES = new Set(['target_changed', 'target_unresolved', 'target_uncovered', 'focus_changed',
+ 'intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired', 'pending_feedback',
+ 'target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed',
+ 'relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable']);
function validationFailureMetadata(value) {
if (VALIDATION_FAILURE_CODES.has(value) || PUBLICATION_CONTRACT_CODES.has(value)) return value;
@@ -150,6 +157,17 @@ function projectNoteExtra(value) {
if (Object.hasOwn(output, 'retryMode')) {
output.retryMode = RECOVERY_RETRY_MODES.has(output.retryMode) ? output.retryMode : 'resume';
}
+ if (Object.hasOwn(output, 'certificationFailure') && !CERTIFICATION_FAILURES.has(output.certificationFailure)) delete output.certificationFailure;
+ if (Object.hasOwn(output, 'predicate') && !ACTION_PREDICATES.has(output.predicate)) delete output.predicate;
+ for (const key of ['monitoringRelaxed', 'focusedTargetAvailable']) {
+ if (Object.hasOwn(output, key) && typeof output[key] !== 'boolean') delete output[key];
+ }
+ for (const key of ['targetCount', 'captureMs']) {
+ if (Object.hasOwn(output, key)) {
+ if (typeof output[key] !== 'number' || !Number.isFinite(output[key]) || output[key] < 0) delete output[key];
+ else output[key] = Math.min(key === 'targetCount' ? 512 : 120000, output[key]);
+ }
+ }
if (Object.hasOwn(output, 'candidateAttempt') && !Number.isSafeInteger(output.candidateAttempt)) {
delete output.candidateAttempt;
}
diff --git a/src/chrome/src/ui/failure-recovery.js b/src/chrome/src/ui/failure-recovery.js
index cfc8567f3..d8d68affb 100644
--- a/src/chrome/src/ui/failure-recovery.js
+++ b/src/chrome/src/ui/failure-recovery.js
@@ -15,13 +15,25 @@ export function failureRecoveryMessage(data = {}) {
return STAGE_MESSAGES[data.stage] || 'The task stopped before it could finish. You can retry from the saved progress.';
}
+export function syncFailureRecoveryMessages(assistantEl) {
+ const normalize = value => String(value || '').replace(/\s+/g, ' ').trim();
+ const textEl = assistantEl?.querySelector('.message-text');
+ const assistantText = normalize(textEl?.innerText || textEl?.textContent);
+ for (const card of assistantEl?.querySelectorAll('.failure-recovery-card') || []) {
+ const message = card.querySelector('.failure-recovery-message');
+ // Keep transcript text for Copy and history; only hide repeated offer copy.
+ if (message) message.hidden = !!assistantText && assistantText === normalize(message.textContent);
+ }
+}
+
export function createFailureRecoveryCard(data, { document = globalThis.document, retryLabel = 'Retry' } = {}) {
if (!data?.recoveryId) return null;
const card = document.createElement('div');
card.className = 'failure-recovery-card';
card.dataset.recoveryId = String(data.recoveryId);
card.dataset.recoveryStage = String(data.stage || 'inference');
- card.dataset.retryMode = data.retryMode === 'publication_without_audit' ? 'publication_without_audit' : 'resume';
+ card.dataset.retryMode = ['publication_without_audit', 'action_monitoring_relaxed'].includes(data.retryMode)
+ ? data.retryMode : 'resume';
card.setAttribute('role', 'alert');
const message = document.createElement('div');
@@ -29,10 +41,12 @@ export function createFailureRecoveryCard(data, { document = globalThis.document
message.textContent = failureRecoveryMessage(data);
card.appendChild(message);
- if (card.dataset.retryMode === 'publication_without_audit') {
+ if (['publication_without_audit', 'action_monitoring_relaxed'].includes(card.dataset.retryMode)) {
const note = document.createElement('div');
note.className = 'failure-recovery-note';
- note.textContent = 'Retry skips the publication audit once for this unchanged draft.';
+ note.textContent = card.dataset.retryMode === 'action_monitoring_relaxed'
+ ? 'Retry relaxes page-change monitoring once for this task.'
+ : 'Retry skips the publication audit once for this unchanged draft.';
card.appendChild(note);
}
if (typeof data.diagnostics === 'string' && data.diagnostics.trim()) {
diff --git a/src/chrome/src/ui/sidepanel.js b/src/chrome/src/ui/sidepanel.js
index a7ef5c930..b59b991f4 100644
--- a/src/chrome/src/ui/sidepanel.js
+++ b/src/chrome/src/ui/sidepanel.js
@@ -39,7 +39,7 @@ import {
} from './chat-history-store.js';
import { historyTextFromElement } from './history-text.js';
import { claimRunError } from './run-error-dedupe.js';
-import { createFailureRecoveryCard } from './failure-recovery.js';
+import { createFailureRecoveryCard, syncFailureRecoveryMessages } from './failure-recovery.js';
import { RUN_CAPTURE_START_ERROR_PREFIX } from '../run-capture.js';
import { runUiUnavailableBeforeSeq, runUiUpdatesForCurrentTask } from '../run-ui-journal.js';
import { formatErrorMessage } from '../error-format.js';
@@ -2950,6 +2950,8 @@ function syncAssistantMessageElementVisibility(msgEl) {
'assistant-awaiting-content',
!assistantMessageHasRenderableContent(msgEl),
);
+ // Terminal and direct responses also write text outside the stream renderer.
+ if (msgEl?.querySelector?.('.failure-recovery-card')) syncFailureRecoveryMessages(msgEl);
}
function syncAssistantMessageVisibility() {
@@ -7604,6 +7606,7 @@ function renderFailureRecoveryCard(assistantEl, data, { tabId = currentTabId, re
const existing = assistantEl.querySelector(`.failure-recovery-card[data-recovery-id="${CSS.escape(recoveryId)}"]`);
if (existing) {
if (existing.dataset.recoveryChosen !== 'true') existing.dataset.recoveryCurrent = 'true';
+ syncFailureRecoveryMessages(assistantEl);
bindFailureRecoveryCard(existing);
syncFailureRecoveryButtons();
return true;
@@ -7616,6 +7619,7 @@ function renderFailureRecoveryCard(assistantEl, data, { tabId = currentTabId, re
card.dataset.foreground = assistantEl.dataset.retryForeground || 'false';
card.dataset.recoveryCurrent = 'true';
assistantEl.querySelector('.message-content')?.appendChild(card);
+ syncFailureRecoveryMessages(assistantEl);
bindFailureRecoveryCard(card);
syncFailureRecoveryButtons();
scrollToBottom();
@@ -7642,6 +7646,7 @@ function rebindFailureRecoveryCards() {
card.dataset.recoveryCurrent = 'unknown';
delete card.dataset.recoveryBusy;
card.removeAttribute('aria-busy');
+ syncFailureRecoveryMessages(card.closest('.message.assistant'));
bindFailureRecoveryCard(card);
}
syncFailureRecoveryButtons();
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index a79251536..3ae746938 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -6519,6 +6519,12 @@ export class Agent extends LoopDetector {
// Claim synchronously before awaiting the external guard so teacher-mode
// startup cannot race a run whose persisted teacher-state check is pending.
this._runningTabs.add(tabId);
+ const monitoringRetry = runOptions.trustedContinuation === true
+ ? this._failureRecovery.takeMonitoringRetry(this._failureRecoveryBinding(tabId), String(runOptions.detachedRequestId || ''))
+ : null;
+ this._monitoringRetryStarts ??= new Map();
+ if (monitoringRetry) this._monitoringRetryStarts.set(tabId, monitoringRetry);
+ else this._monitoringRetryStarts.delete(tabId);
this._failureRecovery.clear(tabId);
this._jevSessions?.delete(tabId);
for (const [id, pending] of this._jevPendingCalls || []) if (pending.tabId === tabId) this._jevPendingCalls.delete(id);
@@ -6543,19 +6549,25 @@ export class Agent extends LoopDetector {
await this.assertRunStartAllowed(tabId, defaultKind, runOptions);
this._throwIfAborted(signal);
if (!this._isStandaloneChatRun(runOptions)) {
- await this._beginPageFeedbackRun(tabId, this._runEntryKind(defaultKind, runOptions), signal);
+ await this._beginPageFeedbackRun(tabId, this._runEntryKind(defaultKind, runOptions), signal, monitoringRetry);
}
this._throwIfAborted(signal);
const feedbackRun = this._pageFeedbackRuns?.get(tabId);
if (feedbackRun && typeof onUpdate === 'function') feedbackRun.onUpdate = onUpdate;
}, 'Browser startup', RUN_STARTUP_TIMEOUT_MS, controller.signal);
} catch (error) {
+ this._restoreFailureRecoveryStart(tabId, monitoringRetry?.requestId, onUpdate, monitoringRetry);
try { this._finishSteeringRun(tabId); } finally { this._releaseRunEntry(tabId); }
throw error;
}
}
_releaseRunEntry(tabId) {
+ const feedbackRun = this._pageFeedbackRuns?.get(tabId);
+ if (feedbackRun?.monitoringRetry && !feedbackRun.monitoringRetry.started) {
+ this._restoreFailureRecoveryStart(tabId, feedbackRun.monitoringRetry.requestId, feedbackRun.onUpdate, feedbackRun.monitoringRetry);
+ }
+ this._monitoringRetryStarts?.delete(tabId);
this._ownedNativeChatReadRepairs.delete(tabId);
this._finishPageFeedbackRun(tabId);
firefoxBidi.stopRun(tabId);
@@ -6604,6 +6616,7 @@ export class Agent extends LoopDetector {
}
const id = String(messageId);
if (!run.acceptedIds.has(id)) {
+ this._invalidateMonitoringRetry?.(tabId, 'task_changed');
run.acceptedIds.add(id);
run.messages.push({ id, text: text.trim() });
// A reviewed plan belongs to the task before this correction. Wake the
@@ -7096,12 +7109,18 @@ export class Agent extends LoopDetector {
if (this._checkAbort(tabId) || this._hasPendingSteering(tabId)) return null;
const binding = this._failureRecoveryBinding(tabId);
const recoveryId = 'recovery_' + secureRandomBase36Token(24);
+ const feedbackRun = this._pageFeedbackRuns?.get(tabId);
+ const monitoringPage = details.stage === 'action_preparation' && details.code === 'action_context_unverified'
+ && this._isActionMode(this._effectiveRunMode(tabId)) && feedbackRun?.frames.get(0)?.token
+ ? { url: feedbackRun.url, documentToken: feedbackRun.frames.get(0).token,
+ documentId: feedbackRun.documents.get(0)?.id || '' } : null;
const descriptor = { recoveryId, stage: details.stage || 'model',
code: details.code || 'request_failed', message: String(details.message || 'The task could not finish.').slice(0,2000),
- retryMode: details.stage === 'publication_audit' ? 'publication_without_audit' : 'resume',
+ retryMode: details.stage === 'publication_audit' ? 'publication_without_audit'
+ : monitoringPage ? 'action_monitoring_relaxed' : 'resume',
retryLabel: 'Retry', giveUpLabel: 'Give up',
...(details.diagnostics ? { diagnostics: String(details.diagnostics).slice(0,6000) } : {}) };
- this._failureRecovery.offer({ ...binding,
+ this._failureRecovery.offer({ ...binding, ...(monitoringPage ? { monitoringPage } : {}),
...(details.publicationKey ? { publicationKey: details.publicationKey,
publicationTaskKey: details.taskKey } : {}) }, descriptor);
onUpdate?.('failure_recovery', descriptor);
@@ -7185,13 +7204,33 @@ export class Agent extends LoopDetector {
return { content: details.message, status: postAction ? 'partial' : 'error' };
}
- requestFailureRecovery(tabId, recoveryId, choice) {
+ _restoreFailureRecoveryStart(tabId, requestId, onUpdate = null, retry = this._failureRecovery.monitoringRetries.get(tabId)) {
+ if (!retry || retry.requestId !== requestId || retry.started || retry.invalidated || retry.restored || this._checkAbort(tabId)) return false;
+ if (this._hasPendingSteering(tabId)) { retry.invalidated = true; return false; }
+ const current = this._failureRecoveryBinding(tabId), binding = retry.offer.binding;
+ if (!['conversationId', 'taskKey', 'taskSignature'].every(key => binding[key] === current[key])) return false;
+ retry.restored = true;
+ const descriptor = { ...retry.offer.descriptor, recoveryId: 'recovery_' + secureRandomBase36Token(24) };
+ this._failureRecovery.offer(binding, descriptor);
+ onUpdate?.('failure_recovery', descriptor);
+ return true;
+ }
+
+ requestFailureRecovery(tabId, recoveryId, choice, requestId = '') {
if (!['retry', 'give_up'].includes(choice)) return { ok: false, error: 'Invalid recovery choice.' };
if (this.isRunning(tabId)) return { ok: false, error: 'Wait for the current run to finish before choosing recovery.' };
const offer = this._failureRecovery.current(this._failureRecoveryBinding(tabId));
if (!offer || offer.descriptor.recoveryId !== recoveryId) {
return { ok: false, error: 'This retry has expired. Start or continue the current task.' };
}
+ if (choice === 'retry' && offer.descriptor.retryMode === 'action_monitoring_relaxed') {
+ if (typeof requestId !== 'string' || !requestId.trim() || requestId.length > 200
+ || !this._failureRecovery.reserveMonitoringRetry(this._failureRecoveryBinding(tabId), recoveryId,
+ requestId, secureRandomBase36Token(24))) {
+ return { ok: false, error: 'This recovery requires its own current retry request.' };
+ }
+ return { ok: true, continue: true, retryMode: offer.descriptor.retryMode };
+ }
if (choice === 'retry' && offer.descriptor.retryMode === 'publication_without_audit') {
const granted = this._grantSocialPublicationAuditRetry(tabId, { ...offer.binding,
taskKey: offer.binding.publicationTaskKey || offer.binding.taskKey });
diff --git a/src/firefox/src/agent/failure-recovery.js b/src/firefox/src/agent/failure-recovery.js
index 820a6f479..8b50277a2 100644
--- a/src/firefox/src/agent/failure-recovery.js
+++ b/src/firefox/src/agent/failure-recovery.js
@@ -1,8 +1,9 @@
// Recovery choices come from the extension UI, never from a model tool call.
export class FailureRecoveryRegistry {
- constructor() { this.offers = new Map(); }
+ constructor() { this.offers = new Map(); this.monitoringRetries = new Map(); }
offer(binding, descriptor) {
+ this.monitoringRetries.delete(binding.tabId);
const offer = { binding: { ...binding }, descriptor: { ...descriptor } };
this.offers.set(binding.tabId, offer);
return { ...offer.descriptor };
@@ -14,12 +15,31 @@ export class FailureRecoveryRegistry {
if (offer.binding.conversationId !== binding.conversationId || offer.binding.taskKey !== binding.taskKey
|| offer.binding.taskSignature !== binding.taskSignature) {
this.offers.delete(binding.tabId);
+ this.monitoringRetries.delete(binding.tabId);
return null;
}
return offer;
}
- clear(tabId) { this.offers.delete(tabId); }
+ reserveMonitoringRetry(binding, recoveryId, requestId, grantToken) {
+ const offer = this.current(binding);
+ if (!requestId || !grantToken || offer?.descriptor.recoveryId !== recoveryId
+ || offer.descriptor.retryMode !== 'action_monitoring_relaxed' || !offer.binding.monitoringPage) return null;
+ const previous = this.monitoringRetries.get(binding.tabId);
+ if (previous) return previous.requestId === requestId ? previous : null;
+ const retry = { offer, requestId, grantToken };
+ this.monitoringRetries.set(binding.tabId, retry);
+ return retry;
+ }
+
+ takeMonitoringRetry(binding, requestId) {
+ const offer = this.current(binding), retry = this.monitoringRetries.get(binding.tabId);
+ if (!retry || retry.offer !== offer || retry.requestId !== requestId) return null;
+ this.clear(binding.tabId);
+ return retry;
+ }
+
+ clear(tabId) { this.offers.delete(tabId); this.monitoringRetries.delete(tabId); }
}
export function modelFailureRecovery(error, { postAction = false, phase = 'model' } = {}) {
diff --git a/src/firefox/src/agent/page-feedback.js b/src/firefox/src/agent/page-feedback.js
index f50dd6686..9dfc0366b 100644
--- a/src/firefox/src/agent/page-feedback.js
+++ b/src/firefox/src/agent/page-feedback.js
@@ -218,11 +218,13 @@ export async function beforePageAgentDispatch(api, tabId, details = {}) {
let acknowledgement;
let preparationAcknowledgement;
const expectedModelSnapshot = owner.expectedModelSnapshot?.();
+ const relaxedMonitorGrantToken = owner.relaxedMonitorGrantToken?.();
try {
if (!details.release && details.fenceOnly && prepareMonitor) {
preparationAcknowledgement = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_prepare',
params: { ...monitorParams, tool: dispatchDetails.tool || dispatchDetails.kind,
allowPassiveRebase: owner.allowPassiveRebase?.() === true,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}),
...(expectedModelSnapshot ? { expectedModelSnapshot } : {}) } }, { frameId: Number(details.frameId) || 0 });
}
acknowledgement = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_dispatch',
@@ -312,11 +314,39 @@ function sameOriginPath(expected, actual) {
}
export const pageFeedbackMethods = {
- async _beginPageFeedbackRun(tabId, kind, signal = null) {
+ _isPassiveMonitoringFeedback(tabId, events, run = this._pageFeedbackRuns?.get(tabId)) {
+ return isPassivePageFeedback(events) || (!!events?.length && !!this._monitoringRetryCurrent(tabId, run)
+ && events.every(event => event.kind === 'dom' && ['page', 'unknown'].includes(event.source)
+ && event.frameId === 0 && !/^(?:base|style|link)(?:$|[\s#.])/.test(event.target || '')));
+ },
+
+ _monitoringRetryCurrent(tabId, run = this._pageFeedbackRuns?.get(tabId)) {
+ const retry = run?.monitoringRetry;
+ if (!retry || retry.invalidated || this._checkAbort(tabId) || this._hasPendingSteering?.(tabId)) return null;
+ const binding = retry.offer.binding, page = binding.monitoringPage;
+ const current = this._failureRecoveryBinding?.(tabId);
+ const matches = this.isRunning(tabId) && page && run.url === page.url
+ && run.frames.get(0)?.token === page.documentToken
+ && (!page.documentId || run.documents.get(0)?.id === page.documentId)
+ && this._steeringRuns?.get(tabId)?.requestId === retry.requestId
+ && ['conversationId', 'taskKey', 'taskSignature'].every(key => current?.[key] === binding[key]);
+ if (!matches) this._invalidateMonitoringRetry(tabId, 'binding_changed');
+ return matches ? retry : null;
+ },
+
+ _invalidateMonitoringRetry(tabId, reason) {
+ const run = this._pageFeedbackRuns?.get(tabId), retry = run?.monitoringRetry || this._monitoringRetryStarts?.get(tabId);
+ if (!retry || retry.invalidated) return;
+ retry.invalidated = true;
+ const runId = this.currentRunId?.get(tabId);
+ if (runId) trace.recordNote(runId, 0, 'monitoring_retry', { status: 'expired', reason });
+ },
+
+ async _beginPageFeedbackRun(tabId, kind, signal = null, monitoringRetry = null) {
this._pageFeedbackRuns ??= new Map();
const run = { token: token(), kind, events: new Map(), frames: new Map(), documents: new Map(),
revision: 0, lastUserAt: 0, lastActivityAt: 0, gestures: new Set(), gestureLeases: new Map(),
- navigation: null, url: '', onUpdate: null };
+ navigation: null, url: '', onUpdate: null, monitoringRetry };
this._pageFeedbackRuns.set(tabId, run);
const assertCurrent = () => {
if (signal) this._throwIfAborted(signal);
@@ -331,6 +361,7 @@ export const pageFeedbackMethods = {
pending: () => this._pageFeedbackBlocksCall(tabId, run.dispatchCall?.name, run.dispatchCall?.args),
allowPassiveRebase: () => run.dispatchCall?.allowPassiveRebase === true,
expectedModelSnapshot: () => run.dispatchCall?.dispatchState?.started ? undefined : run.dispatchCall?.expectedModelSnapshot,
+ relaxedMonitorGrantToken: () => this._monitoringRetryCurrent(tabId, run)?.grantToken,
clearNavigation: () => { run.navigation = null; },
frameForDocument: documentToken => [...run.frames].find(([, frame]) => frame.token === documentToken)?.[0],
dispatched: details => {
@@ -400,7 +431,9 @@ export const pageFeedbackMethods = {
run.frames.set(frameId, { token: documentToken, id: sender.documentId || '', seq: 0,
name: clean(frameName, 256) });
} else previous.name = clean(frameName, 256);
- return { active: true, runToken: run.token, documentToken };
+ const retry = frameId === 0 ? this._monitoringRetryCurrent(tabId, run) : null;
+ return { active: true, runToken: run.token, documentToken,
+ ...(retry ? { relaxedMonitorGrant: { grantToken: retry.grantToken, runToken: run.token, documentToken } } : {}) };
},
observePageFeedback(sender, feedback) {
@@ -479,6 +512,7 @@ export const pageFeedbackMethods = {
const run = this._pageFeedbackRuns?.get(details.tabId);
if (!run || !this.isRunning(details.tabId) || this._checkAbort(details.tabId)) return;
const frameId = Number(details.frameId) || 0;
+ if (frameId === 0) this._invalidateMonitoringRetry(details.tabId, 'navigation');
if (type === 'committed') {
run.frames.delete(frameId);
clearGestureLease(run, frameId);
@@ -526,6 +560,7 @@ export const pageFeedbackMethods = {
_queuePageFeedback(tabId, item) {
const run = this._pageFeedbackRuns?.get(tabId);
if (!run) return;
+ if (!this._isPassiveMonitoringFeedback(tabId, [item], run)) this._invalidateMonitoringRetry(tabId, 'intervention');
run.revision++;
const key = `${item.frameId}:${item.kind}:${item.target || ''}`;
const previous = run.events.get(key);
@@ -794,15 +829,18 @@ export const pageFeedbackMethods = {
// one content task. A snapshot taken after an older observation could bind
// a new recipient to a model decision that still saw the previous one.
const actionCaptureStarted = Date.now();
- let actionCaptureStatus = 'unavailable', actionTargetCount = 0;
+ let actionCaptureStatus = 'unavailable', actionTargetCount = 0, actionMonitoringRelaxed = false;
// Adapter proof is private and app-owned; unsupported pages return no
// preparation. Keep its owner bound across the same capture await.
const nativeChatOwner = this._nativeChatOwnerScope?.(tabId) || '';
+ const monitoringRetry = this._monitoringRetryCurrent(tabId, run);
+ if (monitoringRetry) monitoringRetry.started = true;
if (Array.isArray(messages) && /^https?:\/\//i.test(url) && state.frames.get(0)?.token) {
try {
const api = apiFor();
const capture = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_capture_model',
params: { runToken: run.token, includeTree: true,
+ ...(monitoringRetry ? { relaxedMonitorGrantToken: monitoringRetry.grantToken } : {}),
...(nativeChatOwner ? { nativeChatOwner } : {}),
...(run.uploadObservationScope?.documentToken === state.frames.get(0)?.token
&& run.uploadObservationScope.url === url ? { uploadTargets: run.uploadObservationScope.selectors } : {}),
@@ -820,9 +858,13 @@ export const pageFeedbackMethods = {
&& (!nativeChatOwner || nativeChatOwner === this._nativeChatOwnerScope?.(tabId))
&& this._pageFeedbackRuns?.get(tabId) === run && this._pageFeedbackStateCurrent(tabId, run, state, url)) {
actionCaptureStatus = 'ready';
+ actionMonitoringRelaxed = capture.monitoringRelaxed === true;
+ if (monitoringRetry && capture.monitoringRelaxed === false) this._invalidateMonitoringRetry(tabId, 'grant_expired');
actionTargetCount = Number.isSafeInteger(capture.targetCount) ? Math.max(0, Math.min(512, capture.targetCount)) : 0;
state.actionBinding = { snapshotToken: capture.snapshotToken, runToken: run.token, documentToken: capture.documentToken,
- focusedTargetAvailable: capture.focusedTargetAvailable === true };
+ focusedTargetAvailable: capture.focusedTargetAvailable === true,
+ ...(monitoringRetry && this._monitoringRetryCurrent(tabId, run) === monitoringRetry
+ ? { relaxedMonitorGrantToken: monitoringRetry.grantToken } : {}) };
if (capture.observationScopeCaptured === false) run.actionObservationScope = null;
run.actionCaptureTarget = null;
state.page = run.latestPage = capture.page;
@@ -844,6 +886,7 @@ export const pageFeedbackMethods = {
status: actionCaptureStatus, targetCount: actionTargetCount,
focusedTargetAvailable: state.actionBinding?.focusedTargetAvailable === true,
captureMs: Date.now() - actionCaptureStarted,
+ monitoringRelaxed: actionMonitoringRelaxed,
});
}
},
@@ -885,16 +928,35 @@ export const pageFeedbackMethods = {
const binding = state?.actionBinding;
if (run) { run.actionBindingUnavailable = null; run.actionContextUnverified = false; }
if (!run || !binding || run.modelState !== state || !this._pageFeedbackStateCurrent(tabId, run, state)
- || (run.events.size && !isPassivePageFeedback([...run.events.values()]))) return false;
+ || (run.events.size && !this._isPassiveMonitoringFeedback(tabId, [...run.events.values()], run))) return false;
try {
+ const retry = this._monitoringRetryCurrent(tabId, run);
+ const relaxedMonitorGrantToken = retry && retry.grantToken === binding.relaxedMonitorGrantToken ? retry.grantToken : null;
const response = await apiFor().tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_validate_model',
- params: { ...safeAction(name, args), runToken: run.token, snapshotToken: binding.snapshotToken } }, { frameId: 0 });
+ params: { ...safeAction(name, args), runToken: run.token, snapshotToken: binding.snapshotToken,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}) } }, { frameId: 0 });
const owned = response?.runToken === binding.runToken
&& response.documentToken === binding.documentToken && response.snapshotToken === binding.snapshotToken
&& this._pageFeedbackRuns?.get(tabId) === run && run.modelState === state
&& this._pageFeedbackStateCurrent(tabId, run, state)
- && (!run.events.size || isPassivePageFeedback([...run.events.values()]));
- const valid = owned && response.ready === true;
+ && (!run.events.size || this._isPassiveMonitoringFeedback(tabId, [...run.events.values()], run));
+ const valid = owned && response.ready === true
+ && (response.relaxed !== true || (relaxedMonitorGrantToken && this._monitoringRetryCurrent(tabId, run) === retry));
+ const certificationFailure = ['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit',
+ 'target_uncovered', 'context_uncertified'].includes(response?.certificationFailure) ? response.certificationFailure : null;
+ if (owned) {
+ const runId = this.currentRunId?.get(tabId);
+ if (runId) trace.recordNote(runId, 0, 'page_action_validation', {
+ status: valid ? 'ready' : 'blocked', monitoringRelaxed: valid && response.relaxed === true,
+ ...(certificationFailure ? { certificationFailure } : {}),
+ ...(response.uncertified === true ? { certificationFailure: certificationFailure || 'context_uncertified' } : {}),
+ ...(typeof response.reason === 'string' && ['target_changed', 'target_unresolved', 'target_uncovered',
+ 'focus_changed', 'intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired',
+ 'pending_feedback', 'target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed',
+ 'relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable'].includes(response.reason) ? { predicate: response.reason } : {}),
+ });
+ if (retry && !valid && response.uncertified !== true) this._invalidateMonitoringRetry(tabId, 'target_changed');
+ }
if (valid) {
run.validatedActionBindings ??= new Map();
run.validatedActionBindings.set(`${name}:${JSON.stringify(args)}`, binding.snapshotToken);
@@ -925,8 +987,10 @@ export const pageFeedbackMethods = {
|| !this._pageFeedbackStateCurrent(tabId, run, run.dispatchCall.modelState))) return true;
if (!run?.events.size) return false;
const events = [...run.events.values()];
- if (!isPassivePageFeedback(events) || !this._pageFeedbackStateCurrent(tabId, run, run.modelState)) return true;
- const policy = pageFeedbackCallPolicy(name, args, run.modelState, run.latestPage, events);
+ if (!this._isPassiveMonitoringFeedback(tabId, events, run) || !this._pageFeedbackStateCurrent(tabId, run, run.modelState)) return true;
+ const policyEvents = this._monitoringRetryCurrent(tabId, run)
+ ? events.map(event => event.source === 'unknown' && event.kind === 'dom' ? { ...event, source: 'page' } : event) : events;
+ const policy = pageFeedbackCallPolicy(name, args, run.modelState, run.latestPage, policyEvents);
// Targets need a fresh comparison first; the content monitor then checks
// their immutable node and action context at the actual dispatch boundary.
if (policy.kind === 'bound_target') return run.dispatchCall?.allowPassiveRebase !== true
@@ -960,8 +1024,8 @@ export const pageFeedbackMethods = {
this._flushPageFeedbackMessages(tabId, messages);
const recovery = run?.recoveryResult;
if (!recovery) return null;
- // Retry observes the current page through the normal gates. A context
- // failure never grants the publication-audit bypass reserved for that audit.
+ // A context retry relaxes monitoring through a private one-run grant;
+ // publication-audit recovery remains a separate authorization.
this._offerFailureRecovery?.(tabId, { stage: 'action_preparation', code: recovery.code,
message: recovery.message }, onUpdate);
onUpdate(this._effectiveRunMode?.(tabId, null) === 'ask' ? 'error' : 'warning', { message: recovery.message, code: recovery.code });
@@ -978,6 +1042,21 @@ export const pageFeedbackMethods = {
}
},
+ async _pageFeedbackPreparationFailure(tabId, name, args, state, diagnostic = {}) {
+ const run = this._pageFeedbackRuns?.get(tabId);
+ // Legacy preparation can hit passive churn after an uncertified original
+ // target was allowed with no queued feedback. Preserve the honest reason
+ // only if a new validation still proves that original target unchanged.
+ if (run?.actionContextUnverified && run.modelState === state) {
+ await this._validatePageFeedbackAction(tabId, name, args, state);
+ if (run.actionContextUnverified && this._pageFeedbackStateCurrent(tabId, run, state)) {
+ return { ...pageFeedbackPendingResult(), ...diagnostic, actionContextUnverified: true,
+ errorCode: 'action_context_unverified' };
+ }
+ }
+ return { ...pageFeedbackPendingResult(), ...diagnostic };
+ },
+
_accountPageFeedbackNoDispatch(tabId, name, result) {
if (result?.pageFeedbackPending !== true || result?.noDispatch !== true) return false;
const run = this._pageFeedbackRuns?.get(tabId);
@@ -998,7 +1077,7 @@ export const pageFeedbackMethods = {
const run = this._pageFeedbackRuns?.get(tabId);
if (!run?.events.size) return false;
if (await this._retainIndependentAskAnswer(tabId, [call], false, '', 'preparation')) return false;
- if (!isPassivePageFeedback([...run.events.values()])) {
+ if (!this._isPassiveMonitoringFeedback(tabId, [...run.events.values()], run)) {
if (askAnswerCandidate([call], false, '').trim() && automaticAnswerFeedback([...run.events.values()])
&& this._pageFeedbackAskOwnerCurrent(tabId, run, run.modelState)) {
const messages = [];
@@ -1072,7 +1151,9 @@ export const pageFeedbackMethods = {
&& this._pageFeedbackStateCurrent(tabId, run, state, url)));
const canRetain = (feedback, options) => readonlyResponse
? isPassivePageFeedback(feedback)
- : canRetainPageFeedbackCalls(responseToolCalls, state, page, feedback, options);
+ : canRetainPageFeedbackCalls(responseToolCalls, state, page,
+ this._monitoringRetryCurrent(tabId, run) ? feedback.map(event => event.source === 'unknown' && event.kind === 'dom'
+ ? { ...event, source: 'page' } : event) : feedback, options);
let unverifiedActionContext = false;
let retained = sameDocument && canRetain(events)
&& (!pending.length || canRetain(pending, { pending: true }));
@@ -1222,7 +1303,9 @@ export function installPageFeedback(Agent) {
const operationId = run && mutation ? token() : '';
let noDispatch = false;
const expectedModelSnapshot = modelBound ? modelState.actionBinding.snapshotToken : undefined;
- if (run) run.dispatchCall = { name, args, allowPassiveRebase: false, dispatchState, expectedModelSnapshot, modelState };
+ const relaxedMonitorGrantToken = expectedModelSnapshot && this._monitoringRetryCurrent(tabId, run)?.grantToken;
+ if (run) run.dispatchCall = { name, args, allowPassiveRebase: false, dispatchState, expectedModelSnapshot, modelState,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}) };
if (run && mutation && owner) {
owner.operationId = operationId;
owner.operationFrames.set(operationId, new Set([0]));
@@ -1231,6 +1314,7 @@ export function installPageFeedback(Agent) {
try {
prepared = await apiFor().tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_prepare',
params: { ...safeAction(name, args), runToken: run.token, operationId, allowPassiveRebase: allowPassiveRebase === true,
+ ...(relaxedMonitorGrantToken ? { relaxedMonitorGrantToken } : {}),
...(expectedModelSnapshot ? { expectedModelSnapshot } : {}) } }, { frameId: 0 });
} catch {}
// Certified targets keep their pre-inference footprint across passive
@@ -1270,14 +1354,18 @@ export function installPageFeedback(Agent) {
}
}
if (run && mutation && this._hasPendingPageFeedback(tabId) && result?.noDispatch === true
- && !result.denied && !result.cancelled && !result.outcomeUnknown) return { ...result, pageFeedbackPending: true };
+ && !result.denied && !result.cancelled && !result.outcomeUnknown) {
+ const preparation = await this._pageFeedbackPreparationFailure(tabId, name, args, modelState);
+ return { ...result, pageFeedbackPending: true,
+ ...(preparation.actionContextUnverified ? { actionContextUnverified: true, errorCode: preparation.errorCode } : {}) };
+ }
return result;
} catch (error) {
noDispatch = !dispatchState.started;
if (error?.code === 'page_feedback_pending') {
const facts = preparationVetoDiagnostics(error.pagePreparationVeto);
const diagnostic = Object.keys(facts).length ? { preparationVeto: facts } : {};
- if (!dispatchState.started) return { ...pageFeedbackPendingResult(), ...diagnostic };
+ if (!dispatchState.started) return this._pageFeedbackPreparationFailure(tabId, name, args, modelState, diagnostic);
const interrupted = { success: false, dispatched: true, outcomeUnknown: true, retryable: false,
pageFeedbackPending: true, ...diagnostic, error: 'The browser changed after this action began. Inspect its outcome before attempting another mutation.' };
return this._finalizeToolResultOnce(tabId, name, args, interrupted, dispatchState);
diff --git a/src/firefox/src/background.js b/src/firefox/src/background.js
index 993feed09..b8970efaa 100644
--- a/src/firefox/src/background.js
+++ b/src/firefox/src/background.js
@@ -2186,12 +2186,18 @@ function isPublicationAuditFailureUpdate(update) {
|| (update?.type === 'failure_recovery' && update?.data?.stage === 'publication_audit');
}
+function isActionPreparationFailureUpdate(update) {
+ return update?.type === 'failure_recovery' && update?.data?.stage === 'action_preparation'
+ && ['action_context_unverified', 'page_unstable'].includes(update?.data?.code);
+}
+
function runUpdatesSucceeded(updates = []) {
return !updates.some(update => (
update?.type === 'error' || update?.type === 'quota'
|| isClarificationRequiredRunUpdate(update)
|| isPersistenceDegradedRunUpdate(update)
|| isPublicationAuditFailureUpdate(update)
+ || isActionPreparationFailureUpdate(update)
|| isPlannerRequestFailureUpdate(update)
));
}
@@ -2205,6 +2211,7 @@ function terminalRunUiStatus(content, updates = [], error = null) {
if (updates.some(update => update?.type === 'error'
|| isPlannerRequestFailureUpdate(update) || isPersistenceDegradedRunUpdate(update)
|| isPublicationAuditFailureUpdate(update)
+ || isActionPreparationFailureUpdate(update)
|| update?.type === 'quota')) return 'failed';
if (updates.some(isClarificationRequiredRunUpdate)) return 'clarification_required';
return 'completed';
@@ -2228,6 +2235,7 @@ function askCompletionSucceededForBadge(result, updates = [], error = null) {
update?.type === 'error' || update?.type === 'quota'
|| isPersistenceDegradedRunUpdate(update)
|| isPublicationAuditFailureUpdate(update)
+ || isActionPreparationFailureUpdate(update)
|| update?.type === 'attachment_rejected'
|| update?.type === 'max_steps_reached'
|| update?.error
@@ -2474,6 +2482,7 @@ function launchDetachedRun(action, msg, sender) {
});
entry.promise = task;
task.catch((error) => {
+ agent._restoreFailureRecoveryStart?.(tabId, requestId);
rememberDetachedRunFailure(tabId, requestId, error);
console.warn(`[WebBrain] detached ${action} run failed:`, error);
}).finally(() => {
@@ -2607,9 +2616,14 @@ async function handleFailureRecoveryResponse(msg, sender) {
}
}
assertNoActiveTabRun(tabId);
- const result = agent.requestFailureRecovery(tabId, recoveryId, choice);
+ const result = agent.requestFailureRecovery(tabId, recoveryId, choice, msg.requestId);
if (!result?.ok || choice === 'give_up' || result.continue !== true) return result;
- return launchDetachedRun('continue', msg, sender);
+ try {
+ return launchDetachedRun('continue', msg, sender);
+ } catch (error) {
+ agent._restoreFailureRecoveryStart?.(tabId, msg.requestId);
+ throw error;
+ }
}
async function handleMessage(msg, sender) {
diff --git a/src/firefox/src/content/page-monitor.js b/src/firefox/src/content/page-monitor.js
index b63a8bd02..6f9f255df 100644
--- a/src/firefox/src/content/page-monitor.js
+++ b/src/firefox/src/content/page-monitor.js
@@ -42,6 +42,23 @@
let lastFeedbackDelivery = Promise.resolve();
let agentTurn = null, userTurn = null, lastUserTarget = null;
let identityPreparationCapture = null;
+ // Only the authenticated background state response can establish this grant.
+ // Model/page metadata and ordinary action parameters cannot create one.
+ let relaxedMonitorGrant = null;
+ const revokedRelaxedGrants = new Set();
+ const relaxedObservedBindings = new Set();
+ function revokeRelaxedGrant() {
+ if (relaxedMonitorGrant) revokedRelaxedGrants.add(relaxedMonitorGrant.grantToken);
+ relaxedMonitorGrant = null;
+ relaxedObservedBindings.clear();
+ }
+ function currentRelaxedGrant(token) {
+ return active && !!relaxedMonitorGrant && typeof token === 'string'
+ && token === relaxedMonitorGrant.grantToken && relaxedMonitorGrant.runToken === runToken
+ && relaxedMonitorGrant.documentToken === documentToken
+ && !revokedRelaxedGrants.has(token)
+ && relaxedMonitorGrant.url === location.href && relaxedMonitorGrant.baseURI === document.baseURI;
+ }
let matchedEvents = new WeakMap();
const activePointers = new Set();
const fenceAttribute = 'data-webbrain-page-revision';
@@ -120,6 +137,7 @@
};
const send = event => {
if (!active || disposed) return;
+ if (event.source !== 'agent' && (event.kind !== 'dom' || event.source === 'user')) revokeRelaxedGrant();
publishRevision();
const feedback = { ...event, runToken, documentToken, seq: ++seq, revision };
const pending = event.source !== 'agent';
@@ -157,8 +175,18 @@
|| proof.runToken !== runToken || proof.documentToken !== documentToken
|| proof.userAt !== lastUserAt || proof.interventionRevision !== interventionRevision
|| deepActiveElement() !== op.preparedBinding.target) continue;
- const before = proof.binding, after = actionFootprint(before.target, before.readonlyPreparation);
+ const before = proof.binding, after = before.relaxed
+ ? relaxedTargetBinding(before.target, before.grantToken) : actionFootprint(before.target, before.readonlyPreparation);
if (!after) continue;
+ if (before.relaxed) {
+ const original = proof.viewport;
+ if (!original || !sameRelaxedBinding(before, after)
+ || ![original, viewport].every(value => value.scale === 1 && value.visualWidth === value.width
+ && value.visualHeight === value.height && Math.abs(value.offsetX) < 1 && Math.abs(value.offsetY) < 1)
+ || ['width', 'height', 'visualWidth', 'visualHeight', 'scale', 'dpr'].some(key => original[key] !== viewport[key])) continue;
+ ownedPreparationBindings.set(op.preparedBinding, { ...proof, binding: after, viewportNotificationPending: false });
+ return true;
+ }
const originalContext = JSON.parse(before.context), currentContext = JSON.parse(after.context);
const original = originalContext[2];
if (![original, viewport].every(value => value.scale === 1 && value.visualWidth === value.width
@@ -205,6 +233,7 @@
const interact = (kind, el, { completeGesture = false, ...extra } = {}) => {
const isIgnored = ignored(el);
if (!active || (isIgnored && !completeGesture)) return;
+ revokeRelaxedGrant();
lastUserAt = Date.now(); lastUserTarget = isIgnored ? null : el; revision++;
const marker = {}; userTurn = marker;
setTimeout(() => { if (userTurn === marker) userTurn = null; }, 0);
@@ -371,9 +400,15 @@
const allowedEvent = data.kind === 'input'
? ['keydown', 'beforeinput', 'input', 'change', 'compositionstart', 'compositionend'].includes(event.type)
: data.kind === 'click' ? ['pointerover', 'mouseover', 'pointermove', 'pointerdown', 'mousedown', 'click'].includes(event.type) : false;
- const fresh = op && data.revision === op.preparedRevision && op.preparedRevision === revision
- && !domTimer && !unreported && lastUserAt <= op.userAt;
const signed = op && nativeMarkerSignature(op.nativeSecret, data) === data.signature;
+ const markerRevisionMatches = op && data.revision === op.preparedRevision;
+ const relaxedFresh = markerRevisionMatches && signed && markerTarget && nextSequence && allowedEvent
+ && data.documentToken === documentToken && data.runToken === runToken && op.preparedBinding?.relaxed === true
+ && (flushPendingMutations(op), sampleFormControls(op.target),
+ !nativeReceiverPredicate(op, { kind: data.kind }) && rebasePassivePreparation(op, { documentToken: data.documentToken }));
+ if (relaxedFresh) op.relaxedNativeRevision = data.revision;
+ const fresh = markerRevisionMatches && (op.preparedBinding?.relaxed ? relaxedFresh : (op.preparedRevision === revision
+ && !domTimer && !unreported && lastUserAt <= op.userAt));
if (data.documentToken === documentToken && data.runToken === runToken && op && markerTarget && nextSequence
&& fresh && signed && allowedEvent && ['input', 'click'].includes(data.kind)) {
// The capability is bound to the isolated-world prepared node and consumed once.
@@ -602,6 +637,7 @@
currentViewport(), window.scrollX, window.scrollY]) };
}
function samePreparedBinding(before, after) {
+ if (before?.relaxed || after?.relaxed) return sameRelaxedBinding(before, after);
if (!before || !after || before.target !== after.target || before.scope !== after.scope || before.hit !== after.hit || before.context !== after.context) return false;
return ['ancestors', 'nodes'].every(key => before[key].length === after[key].length
&& before[key].every((item, index) => item.node === after[key][index].node && item.value === after[key][index].value
@@ -616,6 +652,7 @@
? proof.binding : before;
}
function samePreparationSemantics(before, after) {
+ if (before?.relaxed) return sameRelaxedBinding(before, after);
if (!before?.model || !after?.model) return false;
if (![before, after].every(binding => binding.hit === binding.target || binding.target.contains?.(binding.hit))) return false;
const original = JSON.parse(before.context), current = JSON.parse(after.context);
@@ -1511,6 +1548,7 @@
...(readonlyPreparation ? { readonlyPreparation } : {}) };
}
function operationBinding(op, target = op?.preparedBinding?.target) {
+ if (op?.preparedBinding?.relaxed) return relaxedTargetBinding(target, op.preparedBinding.grantToken);
return op?.preparedBinding?.model === true ? actionFootprint(target, op.preparedBinding.readonlyPreparation) : preparedBinding(target);
}
function inputContinuationValue(item, target) {
@@ -1530,6 +1568,8 @@
}
function inputContinuationBindingMatches(op, after) {
const before = preparedBindingFor(op?.preparedBinding);
+ if (before?.relaxed) return op?.dispatched && op.kind === 'input'
+ && sameRelaxedBinding(before, after, true);
if (!op?.dispatched || op.kind !== 'input' || before?.model !== true || !after
|| before.target !== after.target || before.scope !== after.scope || before.hit !== after.hit || before.context !== after.context) return false;
return ['ancestors', 'nodes'].every(key => before[key].length === after[key].length
@@ -1543,7 +1583,11 @@
}
function operationBindingMatches(op, target = op?.preparedBinding?.target) {
const current = operationBinding(op, target);
- return samePreparedBinding(preparedBindingFor(op?.preparedBinding), current) || inputContinuationBindingMatches(op, current);
+ const matches = samePreparedBinding(preparedBindingFor(op?.preparedBinding), current) || inputContinuationBindingMatches(op, current);
+ if (op?.preparedBinding?.relaxed && (!matches || !relaxedActionability(target, { input: kindFor(op.modelBindingTool || op.tool) === 'input' }))) {
+ revokeRelaxedGrant(); return false;
+ }
+ return matches;
}
function modelTarget(params = {}) {
if (!modelActionTools.has(params.tool) || Number.isFinite(params.x) || Number.isFinite(params.y)
@@ -1573,20 +1617,125 @@
value]);
return { value: identity, parent: bindingParent(target), form: target.form || target.closest('form,[role="form"]') };
}
+ function relaxedTargetBinding(target, grantToken) {
+ if (!currentRelaxedGrant(grantToken) || !(target instanceof Element) || !target.isConnected
+ || target.ownerDocument !== document || !modelCandidateIdentity(target)) return null;
+ const ancestors = [];
+ for (let node = bindingParent(target); node; node = bindingParent(node)) {
+ if (ancestors.length >= 64) return null;
+ ancestors.push({ node, value: '' });
+ }
+ // Focus/scroll decoration and extension transport markers do not change
+ // the control's meaning. All other authored attributes remain exact.
+ const attributes = [...target.attributes].filter(attribute => !['class', 'style',
+ 'data-webbrain-native-target', 'data-webbrain-native-action'].includes(attribute.name))
+ .map(attribute => [attribute.name, attribute.value]).sort(([a], [b]) => a.localeCompare(b));
+ const nativeControl = /^(INPUT|TEXTAREA|SELECT|OPTION)$/.test(target.tagName);
+ const fixed = [target.tagName, attributes, target.disabled === true, target.readOnly === true,
+ target.isContentEditable === true, target.type || null];
+ const content = [nativeControl ? target.value : null, target.textContent,
+ nativeControl ? [target.checked === true, target.selected === true, target.selectedIndex ?? null] : null];
+ const form = target.form || target.closest('form,[role="form"]');
+ if (form && (form.attributes.length > 64 || [...form.attributes].some(attribute => attribute.value.length > 8192))) return null;
+ const formIdentity = form ? JSON.stringify([[...form.attributes].filter(attribute => !['class', 'style'].includes(attribute.name))
+ .map(attribute => [attribute.name, attribute.value]).sort(([a], [b]) => a.localeCompare(b)),
+ form.action || null, form.method || null, form.target || null, form.enctype || null]) : null;
+ return { model: true, relaxed: true, grantToken, target, scope: bindingParent(target), form, formIdentity,
+ clickHandler: target.onclick, submitHandler: form?.onsubmit,
+ identity: JSON.stringify([fixed, content]),
+ continuationIdentity: JSON.stringify([fixed, target.isContentEditable ? null : target.textContent]),
+ ancestors, nodes: [], hit: target, context: JSON.stringify([location.href, document.baseURI]) };
+ }
+ function sameRelaxedBinding(before, after, continuingInput = false) {
+ return !!before?.relaxed && !!after?.relaxed && currentRelaxedGrant(before.grantToken)
+ && before.grantToken === after.grantToken && before.target === after.target && before.scope === after.scope
+ && before.form === after.form && before.formIdentity === after.formIdentity && before.context === after.context
+ && before.clickHandler === after.clickHandler && before.submitHandler === after.submitHandler
+ && (continuingInput ? before.continuationIdentity === after.continuationIdentity : before.identity === after.identity)
+ && before.ancestors.length === after.ancestors.length
+ && before.ancestors.every((item, index) => item.node === after.ancestors[index].node);
+ }
+ function sealRelaxedOwnedInput(op) {
+ const original = op?.preparedBinding;
+ if (!original?.relaxed || !op.dispatched || op.kind !== 'input' || op.typingPreparationOnly
+ || !currentRelaxedGrant(original.grantToken) || lastUserAt !== op.userAt) return;
+ const current = relaxedTargetBinding(original.target, original.grantToken);
+ if (!sameRelaxedBinding(original, current, true)) { revokeRelaxedGrant(); return; }
+ // Preserve the immutable inference/dispatch certificate. Only the local
+ // drift baseline advances to this exact owned input's completed body;
+ // otherwise its delayed observer checkpoint would expire the next action.
+ for (const binding of relaxedObservedBindings) if (binding.target === original.target) relaxedObservedBindings.delete(binding);
+ relaxedObservedBindings.add(current);
+ }
+ function finishRelaxedOwnedAction(op) {
+ sealRelaxedOwnedInput(op);
+ if (!op?.preparedBinding?.relaxed || !op.dispatched || op.kind !== 'click'
+ || op.pointerPreparationOnly || op.selectorPreparationOnly || !currentRelaxedGrant(op.preparedBinding.grantToken)
+ || lastUserAt !== op.userAt) return;
+ // A completed click can open its menu or remove its own control. The
+ // old snapshot stays immutable; the next action needs a fresh capture.
+ for (const binding of relaxedObservedBindings) if (binding.target === op.preparedBinding.target) relaxedObservedBindings.delete(binding);
+ }
+ function relaxedSemanticMutation(binding, record) {
+ if (record.type === 'attributes') return [binding.target, binding.form].includes(record.target)
+ && !['class', 'style', 'data-webbrain-native-target', 'data-webbrain-native-action'].includes(record.attributeName)
+ && record.oldValue !== record.target.getAttribute(record.attributeName);
+ if (record.type === 'childList' && [...record.removedNodes].some(node => node === binding.target
+ || node.contains?.(binding.target) || binding.ancestors.some(item => item.node === node))) return true;
+ const node = record.target.nodeType === 1 ? record.target : record.target.parentElement;
+ return ['characterData', 'childList'].includes(record.type)
+ && (node === binding.target || binding.target.contains(node));
+ }
+ function relaxedActionability(target, { hit = false, input = false } = {}) {
+ if (!(target instanceof Element) || !target.isConnected || target.ownerDocument !== document || !visible(target)
+ || !target.matches(modelControlSelector) || target.disabled === true || (input && target.readOnly === true)) return false;
+ for (let node = target; node; node = bindingParent(node)) {
+ const style = getComputedStyle(node);
+ if (node.inert || node.getAttribute('aria-hidden') === 'true' || node.getAttribute('aria-disabled') === 'true'
+ || node.matches('fieldset[disabled]') || style.display === 'none' || ['hidden', 'collapse'].includes(style.visibility)
+ || style.contentVisibility === 'hidden' || Number(style.opacity) === 0) return false;
+ }
+ const rect = target.getBoundingClientRect();
+ if (!(rect.width > 0 && rect.height > 0)) return false;
+ if (!hit) return true;
+ const receiver = deepElementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2);
+ return !!receiver && (receiver === target || target.contains(receiver));
+ }
function modelBindingResult(params = {}) {
const snapshot = modelSnapshots.get(params.snapshotToken || params.expectedModelSnapshot);
if (!active || params.runToken !== runToken || snapshot?.runToken !== runToken) return { reason: 'model_snapshot_invalid' };
if (!snapshot || snapshot.until < Date.now()) return { reason: 'model_snapshot_expired' };
- if (snapshot.interventionRevision !== interventionRevision || snapshot.userAt !== lastUserAt || interventionUnreported)
+ const relaxed = snapshot.relaxedGrantToken && snapshot.relaxedGrantToken === params.relaxedMonitorGrantToken
+ && currentRelaxedGrant(params.relaxedMonitorGrantToken);
+ if (snapshot.relaxedGrantToken && !relaxed) return { reason: 'relaxed_grant_expired' };
+ if (snapshot.userAt !== lastUserAt || (!relaxed && (snapshot.interventionRevision !== interventionRevision || interventionUnreported)))
return { reason: 'intervention_changed' };
- if (pendingDOM && pendingDOM.source !== 'page') return { reason: 'pending_feedback' };
+ if (!relaxed && pendingDOM && pendingDOM.source !== 'page') return { reason: 'pending_feedback' };
const target = modelTarget(params);
- if (!target) return params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason: 'target_unresolved' };
+ if (!target) {
+ if (relaxed) { revokeRelaxedGrant(); return { reason: 'target_unresolved' }; }
+ return params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason: 'target_unresolved' };
+ }
const before = snapshot.targets.get(target), previous = snapshot.uncertified.get(target);
- if (!before && !previous) return { reason: 'target_uncovered' };
- if (params.ref_id && snapshot.hasRefs && snapshot.refs.get(params.ref_id) !== target) return { reason: 'target_changed' };
+ if (!relaxed && !before && !previous) return { reason: 'target_uncovered', certificationFailure: snapshot.captureFailures.get(target) || 'identity_unobserved' };
+ if (params.ref_id && snapshot.hasRefs && snapshot.refs.get(params.ref_id) !== target) {
+ if (relaxed) revokeRelaxedGrant();
+ return { reason: 'target_changed' };
+ }
if (!params.ref_id && !params.selector && !params.nativeTarget && !params.textMatch && modelFocusTools.has(params.tool)
&& snapshot.focusTarget !== target) return { reason: 'focus_changed' };
+ if (relaxed) {
+ const original = snapshot.relaxedTargets.get(target);
+ const current = relaxedTargetBinding(target, params.relaxedMonitorGrantToken);
+ if (!original) return { reason: 'target_uncovered', certificationFailure: snapshot.captureFailures.get(target) || 'identity_unobserved' };
+ if (snapshot.relaxedInvalidated.has(target) || !sameRelaxedBinding(original, current)) { revokeRelaxedGrant(); return { reason: 'target_changed' }; }
+ if (!relaxedActionability(target, { input: kindFor(params.tool) === 'input' })) {
+ revokeRelaxedGrant(); return { reason: 'target_not_actionable' };
+ }
+ for (const binding of relaxedObservedBindings) if (binding.target === target) relaxedObservedBindings.delete(binding);
+ relaxedObservedBindings.add(original);
+ return { binding: original, relaxed: true, certificationFailure: snapshot.captureFailures.get(target) || undefined };
+ }
if (before?.readonlyPreparation && params.tool !== 'click_ax') return {};
if (before) {
const reason = bindingPredicate(preparedBindingFor(before), actionFootprint(target, before.readonlyPreparation));
@@ -1594,7 +1743,7 @@
}
const current = modelCandidateIdentity(target);
return current && previous.value === current.value && previous.parent === current.parent && previous.form === current.form
- ? { uncertified: true } : {};
+ ? { uncertified: true, certificationFailure: snapshot.captureFailures.get(target) || 'owner_unresolved' } : {};
}
function captureModel(params = {}) {
if (!active || params.runToken !== runToken) return { ready: false, runToken, documentToken };
@@ -1700,6 +1849,8 @@
} catch { return { ready: false, runToken, documentToken, reason: 'observation_unavailable' }; }
}
const targets = new Map(), uncertified = new WeakMap(), refs = new Map(), seen = new Set(), publicNodes = new Set();
+ const relaxedGrantToken = page && currentRelaxedGrant(params.relaxedMonitorGrantToken) ? params.relaxedMonitorGrantToken : null;
+ const relaxedTargets = new Map(), relaxedInvalidated = new WeakSet(), captureFailures = new WeakMap();
if (page) for (const { ref, end } of observedRefs) {
if (end > page.pageContent.length) continue;
const node = window.__wb_ax_lookup?.(ref);
@@ -1844,13 +1995,20 @@
// inputs still need the existing structural owner proof.
if (page && !publicNodes.has(node) && !(node.tagName === 'INPUT' && node.type === 'file')) return;
seen.add(node);
- const footprint = targets.size < MODEL_TARGET_LIMIT ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;
- if (footprint && !observedFootprint(footprint)) return;
+ const identity = modelCandidateIdentity(node);
+ if (!identity) captureFailures.set(node, 'capture_limit');
+ if (relaxedGrantToken && publicNodes.has(node) && node.matches(modelControlSelector)) {
+ const binding = relaxedTargetBinding(node, relaxedGrantToken);
+ if (binding) relaxedTargets.set(node, binding);
+ }
+ const limitReached = targets.size >= MODEL_TARGET_LIMIT;
+ const footprint = !limitReached ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;
+ if (footprint && !observedFootprint(footprint)) { captureFailures.set(node, 'identity_unobserved'); return; }
if (!footprint && page && !publicNodes.has(node)) return;
if (footprint) targets.set(node, footprint);
else {
- const identity = modelCandidateIdentity(node);
if (identity) uncertified.set(node, identity);
+ if (!captureFailures.has(node)) captureFailures.set(node, limitReached ? 'capture_limit' : 'owner_unresolved');
}
};
// Recovery observes this target's current semantic owner in the same task
@@ -1888,10 +2046,13 @@
// Metadata-only tests/integration use an existing AX lookup without a new
// tree. Record those refs lazily only when they resolve to an original node.
modelSnapshots.set(snapshotToken, { runToken, until: Date.now() + MODEL_SNAPSHOT_MS, targets, uncertified, refs,
+ relaxedGrantToken, relaxedTargets, relaxedInvalidated, captureFailures,
hasRefs: !!page || !!window.__wbElementMap, focusTarget, userAt: lastUserAt, interventionRevision });
for (const [key, snapshot] of modelSnapshots) if (snapshot.until < Date.now()) modelSnapshots.delete(key);
while (modelSnapshots.size > 2) modelSnapshots.delete(modelSnapshots.keys().next().value);
return { ready: true, runToken, documentToken, snapshotToken, targetCount: targets.size, focusedTargetAvailable,
+ monitoringRelaxed: !!relaxedGrantToken && currentRelaxedGrant(relaxedGrantToken),
+ ...(priorityTarget && captureFailures.has(priorityTarget) ? { certificationFailure: captureFailures.get(priorityTarget) } : {}),
...(params.observationRef ? { observationScopeCaptured } : {}),
...(page ? { page } : {}) };
}
@@ -1903,6 +2064,15 @@
&& !operationBindingMatches(op);
}
function nativeReceiverPredicate(op, params) {
+ if (op?.preparedBinding?.relaxed && (params.ref_id || params.selector || params.textMatch)
+ && resolveTarget(params) !== op.preparedBinding.target) {
+ revokeRelaxedGrant(); return 'target_changed';
+ }
+ if (op?.preparedBinding?.relaxed && ['click', 'input'].includes(params.kind)
+ && params.pointerPreparationOnly !== true
+ && !relaxedActionability(op.preparedBinding.target, { hit: true, input: params.kind === 'input' })) {
+ revokeRelaxedGrant(); return 'target_not_actionable';
+ }
if (!params.nativeTarget || op?.preparedBinding?.model !== true) return '';
// Resolve the actual transport marker alone; a stale marker cannot fall
// back to the requested selector or ref and borrow its certificate.
@@ -1920,12 +2090,13 @@
return 'target_changed';
}
function rebasePassivePreparation(op, params = {}) {
+ const relaxed = op?.preparedBinding?.relaxed === true && currentRelaxedGrant(op.preparedBinding.grantToken);
const continuingInput = op?.dispatched && op.kind === 'input' && op.preparedBinding?.model === true;
const continuingPreparation = op?.dispatched && (op.pointerPreparationOnly === true || op.typingPreparationOnly === true) && op.preparedBinding?.model === true;
if (!active || !op || op.allowPassiveRebase !== true || (op.dispatched && !continuingInput && !continuingPreparation) || op.coordinateSensitive || op.layoutInvalidated
- || op.until < Date.now() || lastUserAt > op.userAt || interventionUnreported
- || op.preparedInterventionRevision !== interventionRevision
- || (pendingDOM && pendingDOM.source !== 'page')
+ || op.until < Date.now() || lastUserAt > op.userAt || (!relaxed && interventionUnreported)
+ || (!relaxed && op.preparedInterventionRevision !== interventionRevision)
+ || (!relaxed && pendingDOM && pendingDOM.source !== 'page')
|| (params.documentToken && params.documentToken !== documentToken)) return false;
const independentNavigation = op.tool === 'navigate';
if (independentNavigation) {
@@ -2199,6 +2370,32 @@
}
function onMutations(records) {
if (!active) return;
+ // Every captured candidate retains its own tombstone, even before the
+ // model selects it. Churn on an unselected control does not expire the
+ // run; selecting a target changed during inference does.
+ if (relaxedMonitorGrant) for (const snapshot of modelSnapshots.values()) {
+ if (!snapshot.relaxedGrantToken || !currentRelaxedGrant(snapshot.relaxedGrantToken)) continue;
+ for (const binding of snapshot.relaxedTargets.values()) if (!snapshot.relaxedInvalidated.has(binding.target)
+ && (records.some(record => relaxedSemanticMutation(binding, record))
+ || (records.some(record => record.type === 'control' && record.target === binding.target)
+ && !sameRelaxedBinding(binding, relaxedTargetBinding(binding.target, binding.grantToken))))) snapshot.relaxedInvalidated.add(binding.target);
+ }
+ // Mutation history also fences an observed node that was changed and then
+ // restored before dispatch. Unrelated live-page churn does not spend the
+ // grant; input continuation can edit only its original editable contents.
+ if (relaxedMonitorGrant) for (const binding of relaxedObservedBindings) {
+ const owned = [...operations.values()].find(op => op.preparedBinding?.relaxed && op.preparedBinding.target === binding.target
+ && op.dispatched && !op.finished && op.kind === 'input' && lastUserAt <= op.userAt);
+ const ownedClick = agentTurn && lastUserAt <= agentTurn.userAt && [...operations.values()].some(op => op.preparedBinding?.relaxed
+ && op.preparedBinding.target === binding.target && op.dispatched && !op.finished && op.kind === 'click'
+ && !op.pointerPreparationOnly && !op.selectorPreparationOnly && op.userAt === agentTurn.userAt);
+ if (ownedClick) continue;
+ const material = records.some(record => relaxedSemanticMutation(binding, record)
+ && !(owned && ['characterData', 'childList'].includes(record.type)));
+ if (material || !sameRelaxedBinding(binding, relaxedTargetBinding(binding.target, binding.grantToken), !!owned)) {
+ revokeRelaxedGrant(); break;
+ }
+ }
let changed = false, attributed = false, source = 'page', target = '';
let diagnosticRecord = null;
const diagnostic = el => {
@@ -2387,9 +2584,20 @@
}
function start(state) {
if (disposed || !state?.active || state.documentToken !== documentToken) return;
- if (active && runToken === state.runToken) return;
+ const grant = state.relaxedMonitorGrant;
+ const authorizedGrant = grant && typeof grant.grantToken === 'string' && grant.grantToken.length >= 16
+ && grant.grantToken.length <= 256 && grant.runToken === state.runToken && grant.documentToken === documentToken
+ && !revokedRelaxedGrants.has(grant.grantToken)
+ ? Object.freeze({ ...grant, url: location.href, baseURI: document.baseURI }) : null;
+ if (active && runToken === state.runToken) {
+ if (relaxedMonitorGrant && (relaxedMonitorGrant.grantToken !== authorizedGrant?.grantToken
+ || !currentRelaxedGrant(relaxedMonitorGrant.grantToken))) revokeRelaxedGrant();
+ relaxedMonitorGrant = authorizedGrant && !revokedRelaxedGrants.has(authorizedGrant.grantToken) ? authorizedGrant : null;
+ return;
+ }
stop();
active = true; runToken = state.runToken; seq = 0; revision = 0; interventionRevision = 0; interventionDiagnostic = ''; publishRevision();
+ relaxedMonitorGrant = authorizedGrant;
observer = new MutationObserver(onMutations);
observeRoot(document);
controlTimer = setInterval(() => { sampleFormControls(); samplePreparedTargets(); }, 250);
@@ -2415,10 +2623,16 @@
event.stopImmediatePropagation();
flushPendingMutations(op);
sampleFormControls(op?.target);
- if (!active || guard.runToken !== runToken || guard.documentToken !== documentToken || guard.revision !== revision
+ const relaxed = op?.preparedBinding?.relaxed === true && guard.revision === op.preparedRevision
+ && guard.runToken === runToken && guard.documentToken === documentToken
+ && !phaseAlreadyConsumed && op.kind === phaseKind
+ && !nativeReceiverPredicate(op, { kind: phaseKind }) && rebasePassivePreparation(op, { documentToken: guard.documentToken });
+ if (relaxed) op.relaxedNativeRevision = guard.revision;
+ if (!active || guard.runToken !== runToken || guard.documentToken !== documentToken || (!relaxed && guard.revision !== revision)
+ || (op?.preparedBinding?.relaxed && !relaxed)
|| !op || phaseAlreadyConsumed || op.kind !== phaseKind
|| (phase === 'input' && elementFor(event) !== op.target && elementFor(event) !== op.focusTarget)
- || op.preparedRevision !== revision || domTimer || unreported || lastUserAt > op.userAt) {
+ || op.preparedRevision !== revision || (!relaxed && (domTimer || unreported)) || lastUserAt > op.userAt) {
event.preventDefault(); return;
}
if (actionPhase) {
@@ -2641,6 +2855,7 @@
listen(window, 'pagehide', () => stop());
}
function stop() {
+ revokeRelaxedGrant();
active = false; runToken = ''; activePointers.clear(); pointerHeld = false; composing = false;
clearTimeout(agentResize?.timer); agentResize = null;
lastUserAt = 0; lastViewport = ''; userTurn = null; lastUserTarget = null; matchedEvents = new WeakMap(); controlIterator = null; preparedOperationCursor = 0;
@@ -2692,8 +2907,9 @@
const bindingResult = expected ? modelBindingResult(params) : null;
const certified = expected ? bindingResult.binding : inherited
&& previous.runToken === runToken && previous.until >= Date.now()
- && previous.preparedInterventionRevision === interventionRevision && lastUserAt === previous.userAt
- && !interventionUnreported && (!pendingDOM || pendingDOM.source === 'page')
+ && lastUserAt === previous.userAt
+ && (previous.preparedBinding.relaxed ? currentRelaxedGrant(previous.preparedBinding.grantToken)
+ : previous.preparedInterventionRevision === interventionRevision && !interventionUnreported && (!pendingDOM || pendingDOM.source === 'page'))
&& !previous.modelBindingInvalid && !previous.layoutInvalidated
&& operationTarget === previous.preparedBinding.target
&& operationBindingMatches(previous, operationTarget) ? previous.preparedBinding : null;
@@ -2717,7 +2933,10 @@
coordinatePoint: coordinateSensitive ? { x: params.x, y: params.y } : null,
coordinateRect: rectFor(coordinateTarget),
until: Date.now() + 30000, dispatched: false, pointerPreparationOnly: false, selectorPreparationOnly: false, consequentialInputPending: false, userAt: lastUserAt, preparedRevision: revision });
- return proofRequired ? { modelBindingValid: !!certified } : {};
+ return proofRequired ? { modelBindingValid: !!certified,
+ ...(certified?.relaxed ? { relaxed: true, bindingKind: 'relaxed_target' } : {}),
+ ...(bindingResult?.reason ? { reason: bindingResult.reason } : {}),
+ ...(bindingResult?.certificationFailure ? { certificationFailure: bindingResult.certificationFailure } : {}) } : {};
}
// The second argument is private producer provenance, never message/model params.
function dispatch(params, selectorPreparationOnly = false, typingPreparationOnly = false) {
@@ -2773,7 +2992,9 @@
const layoutChanged = coordinatePreparationShifted(op);
const bindingChanged = preparedBindingChanged(op);
const rebased = !layoutChanged && rebasePassivePreparation(op, params);
+ const receiverPredicate = nativeReceiverPredicate(op, params);
if (!active || !op || op.modelBindingInvalid || layoutChanged || bindingChanged || (!rebased && (domTimer || unreported))
+ || receiverPredicate
|| (params.element && op.target !== params.element)
|| lastUserAt > op.userAt
|| (Number.isFinite(op.preparedRevision) && op.preparedRevision !== revision)) {
@@ -2794,6 +3015,9 @@
const currentFocus = deepActiveElement();
const rebindingFocus = params.rebindFocus === true;
const fileAssignment = op?.target?.tagName === 'INPUT' && op.target.type === 'file';
+ if (op?.preparedBinding?.relaxed && !relaxedActionability(op.preparedBinding.target, {
+ hit: ['click', 'input'].includes(op.kind), input: op.kind === 'input',
+ })) { revokeRelaxedGrant(); return false; }
// A companion validates once before writing its marker and again before
// the native key. Revalidating the same consumed Tab rebind is safe only
// while its native sequence and exact deeply focused node remain current.
@@ -2805,7 +3029,9 @@
const shifted = coordinatePreparationShifted(op);
const bindingChanged = preparedBindingChanged(op, { allowFocusRebind: focusRebindAvailable });
const preparedRevision = op?.preparedRevision;
- const suppliedRevisionMatches = Number(params.revision) === preparedRevision;
+ const suppliedRevisionMatches = Number(params.revision) === preparedRevision
+ || (op?.preparedBinding?.relaxed && currentRelaxedGrant(op.preparedBinding.grantToken)
+ && Number(params.revision) === op.relaxedNativeRevision);
const rebased = suppliedRevisionMatches && !shifted && rebasePassivePreparation(op, params);
if (!active || params.runToken !== runToken || params.documentToken !== documentToken || !op || op.modelBindingInvalid || shifted || bindingChanged
|| !suppliedRevisionMatches || op.preparedRevision !== revision
@@ -2845,15 +3071,20 @@
const previous = localOperation;
const target = resolveTarget({ ...params, textMatch: action === 'click' ? params.text : undefined })
|| (kindFor(action) === 'input' ? deepActiveElement() : null);
+ const bindingTool = action === 'ax_prepare_field_for_trusted_type' && params.expectedModelSnapshot
+ && currentRelaxedGrant(params.relaxedMonitorGrantToken) ? 'set_field' : action;
const approved = [...operations.values()].reverse().find(op => op.allowPassiveRebase === true && !op.dispatched
&& op.preparedBinding?.target === target && op.until >= Date.now()
- && op.preparedInterventionRevision === interventionRevision && lastUserAt <= op.userAt && !interventionUnreported
- && (!pendingDOM || pendingDOM.source === 'page')
+ && lastUserAt <= op.userAt
+ && (op.preparedBinding.relaxed ? currentRelaxedGrant(op.preparedBinding.grantToken)
+ : op.preparedInterventionRevision === interventionRevision && !interventionUnreported && (!pendingDOM || pendingDOM.source === 'page'))
&& operationBindingMatches(op, target));
prepare({ selector: params.selector, ref_id: params.ref_id, x: params.x, y: params.y,
- textMatch: action === 'click' ? params.text : undefined, tool: action, operationId, runToken,
+ textMatch: action === 'click' ? params.text : undefined, tool: bindingTool, operationId, runToken,
...(params.expectedModelSnapshot !== undefined || approved?.expectedModelSnapshot !== undefined
? { expectedModelSnapshot: params.expectedModelSnapshot ?? approved.expectedModelSnapshot } : {}),
+ ...(approved?.preparedBinding?.relaxed ? { relaxedMonitorGrantToken: approved.preparedBinding.grantToken } : {}),
+ ...(bindingTool !== action ? { relaxedMonitorGrantToken: params.relaxedMonitorGrantToken } : {}),
allowPassiveRebase: params.allowPassiveRebase === true || !!approved });
if (approved?.preparedBinding?.model === true) {
const local = operations.get(operationId);
@@ -2873,7 +3104,7 @@
if (!params._bidiPrepare) beforeLocalDispatch({ preparation: true });
return () => {
const op = operations.get(operationId);
- if (op) { op.until = Date.now() + 200; op.finished = true; }
+ if (op) { finishRelaxedOwnedAction(op); op.until = Date.now() + 200; op.finished = true; }
localOperation = previous;
};
}
@@ -2885,8 +3116,9 @@
const layoutChanged = coordinatePreparationShifted(operation);
const bindingChanged = preparedBindingChanged(operation);
const rebased = !layoutChanged && rebasePassivePreparation(operation, { element: target });
+ const receiverPredicate = !preparation ? nativeReceiverPredicate(operation, { kind: kind || localOperation.kind }) : '';
if (rebased) localOperation.revision = operation.preparedRevision;
- if (operation?.modelBindingInvalid || layoutChanged || bindingChanged || (!rebased && (domTimer || unreported)) || lastUserAt > localOperation.userAt || revision !== localOperation.revision) {
+ if (operation?.modelBindingInvalid || layoutChanged || bindingChanged || receiverPredicate || (!rebased && (domTimer || unreported)) || lastUserAt > localOperation.userAt || revision !== localOperation.revision) {
const error = new Error('Browser changed during action preparation. Re-observe before acting.');
error.code = 'page_feedback_pending'; error.dispatched = localOperation.started === true;
error.pagePreparationVeto = preparationVetoFacts(operation, { layoutChanged, bindingChanged, target });
@@ -2918,7 +3150,7 @@
beforeLocalDispatch({ preparation: true, kind, target });
const before = preparedBindingFor(op.preparedBinding);
const previousProof = ownedPreparationBindings.get(op.preparedBinding);
- const previousScrollX = window.scrollX, previousScrollY = window.scrollY;
+ const previousScrollX = window.scrollX, previousScrollY = window.scrollY, preparationViewport = currentViewport();
const preparedIntervention = interventionRevision, preparedUserAt = lastUserAt, preparedRun = runToken;
dispatch({ ...localOperation, runToken, kind, element: target, scrollIntoView: kind === 'scroll' || op.windowScroll,
addScrollTarget: true,
@@ -2928,9 +3160,9 @@
try {
const result = callback();
flushPendingMutations(op); sampleFormControls(target);
- const after = actionFootprint(target, before.readonlyPreparation);
+ const after = before.relaxed ? relaxedTargetBinding(target, before.grantToken) : actionFootprint(target, before.readonlyPreparation);
const changedAuthority = !active || runToken !== preparedRun || lastUserAt !== preparedUserAt
- || interventionRevision !== preparedIntervention || interventionUnreported;
+ || (before.relaxed ? !currentRelaxedGrant(before.grantToken) : interventionRevision !== preparedIntervention || interventionUnreported);
const changedFocus = kind === 'focus' && deepActiveElement() !== target;
if (changedAuthority || changedFocus || !samePreparationSemantics(before, after)) {
const error = new Error('Browser changed during action preparation. Re-observe before acting.');
@@ -2942,6 +3174,7 @@
// derived geometry proof for this original node is shared with its owner.
ownedPreparationBindings.set(op.preparedBinding, { binding: after, runToken, documentToken,
userAt: lastUserAt, interventionRevision,
+ ...(before.relaxed ? { viewport: preparationViewport } : {}),
scrollPreparationUntil: kind === 'scroll' && (previousScrollX !== window.scrollX || previousScrollY !== window.scrollY)
? Date.now() + 200 : previousProof?.scrollPreparationUntil || 0,
viewportNotificationPending: kind === 'scroll' && (previousScrollX !== window.scrollX || previousScrollY !== window.scrollY)
@@ -3020,6 +3253,8 @@
flushPendingMutations(); sampleFormControls();
const result = modelBindingResult(params);
respond({ ready: !!result.binding, ...(result.uncertified ? { uncertified: true } : {}),
+ ...(result.relaxed ? { relaxed: true, bindingKind: 'relaxed_target' } : {}),
+ ...(result.certificationFailure ? { certificationFailure: result.certificationFailure } : {}),
...(result.reason ? { reason: result.reason } : {}), runToken, documentToken, snapshotToken: params.snapshotToken });
}
else if (msg.action === 'page_monitor_prepare') { respond({ ready: true, ...prepare(msg.params || {}) }); }
@@ -3077,7 +3312,7 @@
&& (!active || params.runToken !== runToken || params.documentToken !== documentToken || !op)) {
respond({ ready: false }); return;
}
- if (op) { op.until = Date.now() + 200; op.finished = true; }
+ if (op) { finishRelaxedOwnedAction(op); op.until = Date.now() + 200; op.finished = true; }
respond({ ready: true });
}
};
diff --git a/src/firefox/src/trace/privacy.js b/src/firefox/src/trace/privacy.js
index 9eb0b0574..816d70045 100644
--- a/src/firefox/src/trace/privacy.js
+++ b/src/firefox/src/trace/privacy.js
@@ -40,6 +40,7 @@ const NOTE_METADATA_FIELDS = [
'success', 'approved', 'outcome', 'verification', 'count',
'adapter', 'revision', 'notesInjected', 'workflowSchema', 'job', 'template',
'validationFailure', 'candidateAttempt', 'stage', 'retryMode',
+ 'certificationFailure', 'monitoringRelaxed', 'predicate', 'targetCount', 'focusedTargetAvailable', 'captureMs',
];
// Checker diagnostics are finite protocol labels, never arbitrary model text
@@ -73,7 +74,13 @@ const RECOVERY_STAGES = new Set([
'completion_verification', 'verification', 'action_preparation',
'response_generation', 'browser_interaction',
]);
-const RECOVERY_RETRY_MODES = new Set(['resume', 'publication_without_audit', 'publication_audit_bypass']);
+const RECOVERY_RETRY_MODES = new Set(['resume', 'publication_without_audit', 'publication_audit_bypass', 'action_monitoring_relaxed']);
+const CERTIFICATION_FAILURES = new Set(['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit',
+ 'target_uncovered', 'context_uncertified']);
+const ACTION_PREDICATES = new Set(['target_changed', 'target_unresolved', 'target_uncovered', 'focus_changed',
+ 'intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired', 'pending_feedback',
+ 'target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed',
+ 'relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable']);
function validationFailureMetadata(value) {
if (VALIDATION_FAILURE_CODES.has(value) || PUBLICATION_CONTRACT_CODES.has(value)) return value;
@@ -150,6 +157,17 @@ function projectNoteExtra(value) {
if (Object.hasOwn(output, 'retryMode')) {
output.retryMode = RECOVERY_RETRY_MODES.has(output.retryMode) ? output.retryMode : 'resume';
}
+ if (Object.hasOwn(output, 'certificationFailure') && !CERTIFICATION_FAILURES.has(output.certificationFailure)) delete output.certificationFailure;
+ if (Object.hasOwn(output, 'predicate') && !ACTION_PREDICATES.has(output.predicate)) delete output.predicate;
+ for (const key of ['monitoringRelaxed', 'focusedTargetAvailable']) {
+ if (Object.hasOwn(output, key) && typeof output[key] !== 'boolean') delete output[key];
+ }
+ for (const key of ['targetCount', 'captureMs']) {
+ if (Object.hasOwn(output, key)) {
+ if (typeof output[key] !== 'number' || !Number.isFinite(output[key]) || output[key] < 0) delete output[key];
+ else output[key] = Math.min(key === 'targetCount' ? 512 : 120000, output[key]);
+ }
+ }
if (Object.hasOwn(output, 'candidateAttempt') && !Number.isSafeInteger(output.candidateAttempt)) {
delete output.candidateAttempt;
}
diff --git a/src/firefox/src/ui/failure-recovery.js b/src/firefox/src/ui/failure-recovery.js
index cfc8567f3..d8d68affb 100644
--- a/src/firefox/src/ui/failure-recovery.js
+++ b/src/firefox/src/ui/failure-recovery.js
@@ -15,13 +15,25 @@ export function failureRecoveryMessage(data = {}) {
return STAGE_MESSAGES[data.stage] || 'The task stopped before it could finish. You can retry from the saved progress.';
}
+export function syncFailureRecoveryMessages(assistantEl) {
+ const normalize = value => String(value || '').replace(/\s+/g, ' ').trim();
+ const textEl = assistantEl?.querySelector('.message-text');
+ const assistantText = normalize(textEl?.innerText || textEl?.textContent);
+ for (const card of assistantEl?.querySelectorAll('.failure-recovery-card') || []) {
+ const message = card.querySelector('.failure-recovery-message');
+ // Keep transcript text for Copy and history; only hide repeated offer copy.
+ if (message) message.hidden = !!assistantText && assistantText === normalize(message.textContent);
+ }
+}
+
export function createFailureRecoveryCard(data, { document = globalThis.document, retryLabel = 'Retry' } = {}) {
if (!data?.recoveryId) return null;
const card = document.createElement('div');
card.className = 'failure-recovery-card';
card.dataset.recoveryId = String(data.recoveryId);
card.dataset.recoveryStage = String(data.stage || 'inference');
- card.dataset.retryMode = data.retryMode === 'publication_without_audit' ? 'publication_without_audit' : 'resume';
+ card.dataset.retryMode = ['publication_without_audit', 'action_monitoring_relaxed'].includes(data.retryMode)
+ ? data.retryMode : 'resume';
card.setAttribute('role', 'alert');
const message = document.createElement('div');
@@ -29,10 +41,12 @@ export function createFailureRecoveryCard(data, { document = globalThis.document
message.textContent = failureRecoveryMessage(data);
card.appendChild(message);
- if (card.dataset.retryMode === 'publication_without_audit') {
+ if (['publication_without_audit', 'action_monitoring_relaxed'].includes(card.dataset.retryMode)) {
const note = document.createElement('div');
note.className = 'failure-recovery-note';
- note.textContent = 'Retry skips the publication audit once for this unchanged draft.';
+ note.textContent = card.dataset.retryMode === 'action_monitoring_relaxed'
+ ? 'Retry relaxes page-change monitoring once for this task.'
+ : 'Retry skips the publication audit once for this unchanged draft.';
card.appendChild(note);
}
if (typeof data.diagnostics === 'string' && data.diagnostics.trim()) {
diff --git a/src/firefox/src/ui/sidepanel.js b/src/firefox/src/ui/sidepanel.js
index 3b4944bce..c0a81ac49 100644
--- a/src/firefox/src/ui/sidepanel.js
+++ b/src/firefox/src/ui/sidepanel.js
@@ -39,7 +39,7 @@ import {
} from './chat-history-store.js';
import { historyTextFromElement } from './history-text.js';
import { claimRunError } from './run-error-dedupe.js';
-import { createFailureRecoveryCard } from './failure-recovery.js';
+import { createFailureRecoveryCard, syncFailureRecoveryMessages } from './failure-recovery.js';
import { RUN_CAPTURE_START_ERROR_PREFIX } from '../run-capture.js';
import { runUiUnavailableBeforeSeq, runUiUpdatesForCurrentTask } from '../run-ui-journal.js';
import { formatErrorMessage } from '../error-format.js';
@@ -2219,6 +2219,8 @@ function syncAssistantMessageElementVisibility(msgEl) {
'assistant-awaiting-content',
!assistantMessageHasRenderableContent(msgEl),
);
+ // Terminal and direct responses also write text outside the stream renderer.
+ if (msgEl?.querySelector?.('.failure-recovery-card')) syncFailureRecoveryMessages(msgEl);
}
function syncAssistantMessageVisibility() {
@@ -7450,6 +7452,7 @@ function renderFailureRecoveryCard(assistantEl, data, { tabId = currentTabId, re
const existing = assistantEl.querySelector(`.failure-recovery-card[data-recovery-id="${CSS.escape(recoveryId)}"]`);
if (existing) {
if (existing.dataset.recoveryChosen !== 'true') existing.dataset.recoveryCurrent = 'true';
+ syncFailureRecoveryMessages(assistantEl);
bindFailureRecoveryCard(existing);
syncFailureRecoveryButtons();
return true;
@@ -7462,6 +7465,7 @@ function renderFailureRecoveryCard(assistantEl, data, { tabId = currentTabId, re
card.dataset.foreground = assistantEl.dataset.retryForeground || 'false';
card.dataset.recoveryCurrent = 'true';
assistantEl.querySelector('.message-content')?.appendChild(card);
+ syncFailureRecoveryMessages(assistantEl);
bindFailureRecoveryCard(card);
syncFailureRecoveryButtons();
scrollToBottom();
@@ -7488,6 +7492,7 @@ function rebindFailureRecoveryCards() {
card.dataset.recoveryCurrent = 'unknown';
delete card.dataset.recoveryBusy;
card.removeAttribute('aria-busy');
+ syncFailureRecoveryMessages(card.closest('.message.assistant'));
bindFailureRecoveryCard(card);
}
syncFailureRecoveryButtons();
diff --git a/test/failure-recovery-ui.mjs b/test/failure-recovery-ui.mjs
index 3d2622a09..482807f2b 100644
--- a/test/failure-recovery-ui.mjs
+++ b/test/failure-recovery-ui.mjs
@@ -69,6 +69,209 @@ for (const [build, engine] of [['chrome', chromium], ['firefox', firefox]]) {
await page.waitForFunction(() => calls.length === 2);
assert.deepEqual(await page.evaluate(() => calls[1]), { action: 'failure_recovery_response', payload: { tabId: 7, recoveryId: 'verification-offer', choice: 'give_up' } });
assert.equal(await page.locator('[data-recovery-id="verification-offer"] .failure-recovery-note').count(), 0);
+
+ await page.evaluate(() => {
+ renderFailureRecoveryCard(currentAssistantEl, {
+ recoveryId: 'monitoring-offer', stage: 'browser_interaction', retryMode: 'action_monitoring_relaxed',
+ }, { tabId: 7 });
+ });
+ const monitoringCard = page.locator('[data-recovery-id="monitoring-offer"]');
+ assert.equal(await monitoringCard.getAttribute('data-retry-mode'), 'action_monitoring_relaxed');
+ assert.equal(await monitoringCard.locator('.failure-recovery-note').textContent(), 'Retry relaxes page-change monitoring once for this task.');
+ assert.equal(await page.evaluate(() => calls.length), 2, 'Displaying the retry note must not change monitoring automatically');
+ await monitoringCard.locator('[data-recovery-choice="retry"]').click();
+ assert.deepEqual(await page.evaluate(() => calls[2]), { type: 'retry', options: { failureRecoveryId: 'monitoring-offer', mode: 'act', foreground: false } });
+ await page.evaluate(() => settleRetry(true));
+ await page.waitForFunction(() => document.querySelector('[data-recovery-id="monitoring-offer"]').dataset.recoveryChosen === 'true');
+ } finally { await browser.close(); }
+ });
+
+ test(`${build}: recovery explanations stay singular through live updates, terminal responses, and restoration`, async t => {
+ const source = read(build, 'src/ui/sidepanel.js');
+ const helperStart = source.indexOf('const ASSISTANT_RENDERABLE_ELEMENT_SELECTOR =');
+ const observerStart = source.indexOf('const persistObserver =', helperStart);
+ const observerEnd = source.indexOf('\n});', observerStart) + '\n});'.length;
+ const observeCall = source.match(/persistObserver\.observe\(messagesEl, \{[^\n]+\}\);/)?.[0];
+ assert.ok(helperStart >= 0 && observerStart > helperStart && observerEnd > observerStart && observeCall);
+ const functions = [
+ 'syncFailureRecoveryButtons', 'bindFailureRecoveryCard', 'renderFailureRecoveryCard',
+ 'reconcileFailureRecoveryCards', 'rebindFailureRecoveryCards', 'applyActiveRunState',
+ 'getStreamedAssistantText', 'hasStreamedAssistantText', 'clearStreamedAssistantText',
+ 'renderStreamedAssistantMarkdownNow', 'scheduleStreamedAssistantMarkdownRender',
+ 'clearAssistantTextStreamState', 'renderAssistantTextUpdate', 'isStoppedByUserStatus',
+ 'handleAgentUpdateMessage', 'addMessageCopyButton', 'getMessageCopyText',
+ 'roleFromMessageElement', 'normalizeHistoryText', 'extractChatHistoryMessages', 'formatMarkdown',
+ ].map(name => extract(source, name)).join('\n');
+ const send = extract(source, 'sendMessage');
+ const responseStart = send.indexOf(' if (renderToCurrentTab && currentTabId === tabId && isTabAbortRequested(tabId))');
+ const responseEnd = send.indexOf('\n } catch (e) {', responseStart);
+ const continuation = extract(source, 'continueAgent');
+ const continuationStart = continuation.indexOf(' if (currentTabId === tabId && res?.content && assistantEl)');
+ const continuationEnd = continuation.indexOf('\n } catch (e) {', continuationStart);
+ assert.ok(responseStart >= 0 && responseEnd > responseStart && continuationStart >= 0 && continuationEnd > continuationStart);
+ const friendly = 'Failed while contacting the model. You can retry from the current state.';
+ const browser = await engine.launch({ headless: true });
+ try {
+ async function withPage(run) {
+ const page = await browser.newPage();
+ try {
+ await page.setContent(`<style>${read(build, 'styles/sidepanel.css')}</style><main id="messages"><div class="message assistant" data-run-mode="act" data-run-request-id="failed-request"><div class="message-content"><div class="message-text"></div></div></div></main>`);
+ const unexport = source => source.replace(/^export /gm, '');
+ await page.addScriptTag({ content: `
+ ${unexport(read(build, 'src/ui/failure-recovery.js'))}
+ ${unexport(read(build, 'src/ui/utils.js'))}
+ ${unexport(read(build, 'src/ui/markdown-render.js'))}
+ ${unexport(read(build, 'src/ui/markdown-link.js'))}
+ ${unexport(read(build, 'src/ui/history-text.js'))}
+ var messagesEl = document.querySelector('#messages'), currentAssistantEl = document.querySelector('.assistant');
+ var currentTabId = 7, renderedTabId = 7, agentMode = 'act', isProcessing = false;
+ var verboseMode = false, compactProgressVisible = true;
+ var processingTabs = new Set(), localRunRequestIds = new Map();
+ var clearedConversationRunRequestIds = new Set(), cancelledRunRecoveryRequestIds = new Set();
+ var streamedAssistantTextByEl = new WeakMap(), streamedAssistantRenderFrameByEl = new WeakMap();
+ var offer = { recoveryId: 'one-offer', stage: 'inference', message: ${JSON.stringify(friendly)} };
+ function sameTabId(a, b) { return Number(a) === Number(b); }
+ function isConversationClearInProgress() { return false; }
+ function t(key) { return key === 'sp.retry' ? 'Retry' : key; }
+ function scrollToBottom() {}
+ function schedulePersist() {}
+ function scheduleMathRender() {}
+ function bindMessageCopyButton() {}
+ function showActivity() {}
+ function hideActivity() {}
+ function hideRecommendedActions() {}
+ function startThinkingActivity() {}
+ function markLastStepFailed() {}
+ function finishResponseRetry() {}
+ function parseCostAllowanceError() { return null; }
+ function renderCostAllowanceError() { return false; }
+ function renderSubscribeError() { return false; }
+ function isTabAbortRequested() { return false; }
+ function ensureCurrentRunAssistant() { return currentAssistantEl; }
+ function setMessageCreatedAt() {}
+ function messageCreatedAt() { return 1; }
+ function messageAttachmentMetadata() { return []; }
+ function finalizeSteps() {}
+ function reconcileRunMessageAttachmentState() {}
+ async function reconcilePersistedStagedScreenshots() {}
+ async function restorePendingCurrentClarification() { return false; }
+ function invalidatePlanReviewCards() {}
+ function setPlanReviewAwaiting() {}
+ function clearPlanReviewActiveRun() {}
+ function isPlanReviewTimeoutTerminal() { return false; }
+ function setTabProcessing(tabId, value) {
+ isProcessing = value;
+ if (value) processingTabs.add(tabId); else processingTabs.delete(tabId);
+ syncFailureRecoveryButtons();
+ }
+ function setTabAbortRequested() {}
+ function syncSendButtonState() {}
+ function runUiUnavailableBeforeSeq() { return 0; }
+ function isTerminalRunUiStatus(status) { return ['completed', 'stopped', 'cancelled', 'failed'].includes(status); }
+ async function sendToBackground() { return { ok: true }; }
+ function activeRetryPayloadForRequest() { return null; }
+ function retryPayloadForRunAssistant() { return null; }
+ function looksLikeRawToolCallText() { return false; }
+ ${source.slice(helperStart, observerEnd)}
+ ${functions}
+ ${observeCall}
+ function update(type, data = {}) {
+ handleAgentUpdateMessage({ tabId: 7, requestId: 'failed-request', type, data });
+ }
+ function applyReturnedResponse(res) {
+ const renderToCurrentTab = true, tabId = 7, assistantEl = currentAssistantEl, modeForSend = 'act', retryPayload = null;
+ ${send.slice(responseStart, responseEnd)}
+ }
+ function applyReturnedContinuation(res) {
+ const tabId = 7, assistantEl = currentAssistantEl, modeForSend = 'act';
+ ${continuation.slice(continuationStart, continuationEnd)}
+ }
+ ` });
+ await run(page);
+ } finally { await page.close(); }
+ }
+
+ async function assertSingleExplanation(page) {
+ await page.waitForFunction(() => document.querySelector('.failure-recovery-message')?.hidden === true);
+ assert.equal(await page.locator('.failure-recovery-card').count(), 1);
+ assert.equal(await page.locator('.assistant .message-text:visible, .failure-recovery-message:visible').count(), 1);
+ assert.equal(await page.locator('.failure-recovery-action:visible').count(), 2);
+ assert.equal(await page.evaluate(() => getMessageCopyText(document.querySelector('.msg-copy-btn')).trim()), friendly);
+ assert.deepEqual(await page.evaluate(() => extractChatHistoryMessages().map(message => message.text)), [friendly]);
+ }
+
+ await t.test('text before the offer keeps one explanation and repeated offers stay idempotent', () => withPage(async page => {
+ await page.evaluate(() => {
+ update('text', { content: offer.message });
+ update('failure_recovery', offer);
+ update('failure_recovery', offer);
+ });
+ await assertSingleExplanation(page);
+ }));
+
+ await t.test('the offer retains its fallback for empty or distinct assistant output', () => withPage(async page => {
+ await page.evaluate(() => update('failure_recovery', offer));
+ assert.equal(await page.locator('.failure-recovery-message').isVisible(), true);
+ await page.evaluate(() => update('text', { content: offer.message + '\n', replace: true }));
+ await assertSingleExplanation(page);
+ await page.evaluate(() => update('text', { content: 'The draft is ready for review.', replace: true }));
+ await page.waitForFunction(() => document.querySelector('.failure-recovery-message').hidden === false);
+ assert.equal(await page.locator('.assistant .message-text').textContent(), 'The draft is ready for review.');
+ assert.equal(await page.locator('.failure-recovery-message').isVisible(), true);
+ await page.evaluate(() => update('text', { content: '', replace: true }));
+ assert.equal(await page.locator('.assistant .message-text').textContent(), '');
+ assert.equal(await page.locator('.failure-recovery-message').isVisible(), true);
+ }));
+
+ for (const terminalPath of ['run_complete', 'chat_response', 'continuation_response', 'stream']) {
+ await t.test(`${terminalPath} merges terminal failure copy with the existing offer`, () => withPage(async page => {
+ await page.evaluate(terminalPath => {
+ update('failure_recovery', offer);
+ if (terminalPath === 'run_complete') update('run_complete', { status: 'failed', finalContent: offer.message });
+ if (terminalPath === 'chat_response') applyReturnedResponse({ content: offer.message });
+ if (terminalPath === 'continuation_response') applyReturnedContinuation({ content: offer.message });
+ if (terminalPath === 'stream') update('text_delta', { content: offer.message });
+ }, terminalPath);
+ if (terminalPath === 'stream') {
+ await page.waitForFunction(() => document.querySelector('.failure-recovery-message').hidden === true);
+ await page.evaluate(() => update('run_complete', { status: 'failed', finalContent: offer.message }));
+ }
+ await assertSingleExplanation(page);
+ }));
+ }
+
+ await t.test('legacy saved duplicates normalize on rebind and remain singular after reconnect', () => withPage(async page => {
+ const normalized = await page.evaluate(() => {
+ update('text', { content: offer.message });
+ update('failure_recovery', offer);
+ document.querySelector('.failure-recovery-message').hidden = false;
+ messagesEl.innerHTML = messagesEl.innerHTML;
+ currentAssistantEl = document.querySelector('.assistant');
+ messagesEl.querySelectorAll('[data-bound]').forEach(node => delete node.dataset.bound);
+ rebindFailureRecoveryCards();
+ return document.querySelector('.failure-recovery-message').hidden;
+ });
+ assert.equal(normalized, true, 'Rebinding normalizes restored copy before the observer runs');
+ assert.equal(await page.locator('.failure-recovery-action:disabled').count(), 2);
+ await page.evaluate(async () => {
+ const state = { running: false, runUi: { requestId: 'failed-request', status: 'failed', seq: 4, ackedSeq: 4, events: [], finalContent: offer.message }, pendingFailureRecovery: offer };
+ await applyActiveRunState(7, state);
+ await applyActiveRunState(7, state);
+ });
+ assert.equal(await page.locator('.failure-recovery-action:disabled').count(), 0);
+ await assertSingleExplanation(page);
+ }));
+
+ await t.test('reconnect restores missing final text and recovery from a terminal snapshot', () => withPage(async page => {
+ await page.evaluate(async () => {
+ await applyActiveRunState(7, {
+ running: false,
+ runUi: { requestId: 'failed-request', status: 'failed', seq: 4, ackedSeq: 4, events: [], finalContent: offer.message },
+ pendingFailureRecovery: offer,
+ });
+ });
+ await assertSingleExplanation(page);
+ }));
} finally { await browser.close(); }
});
diff --git a/test/failure-recovery.mjs b/test/failure-recovery.mjs
index c79a7b86e..692361c21 100644
--- a/test/failure-recovery.mjs
+++ b/test/failure-recovery.mjs
@@ -38,6 +38,124 @@ for (const browser of ['chrome', 'firefox']) {
return social;
}
+ function monitoringFixture(agent) {
+ agent.conversationModes.set(tabId, 'act');
+ agent._pageFeedbackRuns ??= new Map();
+ agent._pageFeedbackRuns.set(tabId, { url: 'https://owned.example.test/', token: 'old-run',
+ frames: new Map([[0, { token: 'owned-document', id: 'browser-document' }]]),
+ documents: new Map([[0, { id: 'browser-document' }]]), events: new Map(), gestures: new Set(), gestureLeases: new Map() });
+ return agent._offerFailureRecovery(tabId, { stage: 'action_preparation', code: 'action_context_unverified',
+ message: 'The current control has no certified owner.' });
+ }
+
+ test(`${browser}: context Retry reserves a private grant for exactly its accepted request`, () => {
+ const agent = setup(), offer = monitoringFixture(agent);
+ assert.equal(offer.retryMode, 'action_monitoring_relaxed');
+ assert.equal(agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry').ok, false);
+ assert.equal(agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry', 'owned-retry').ok, true);
+ const retry = agent._failureRecovery.monitoringRetries.get(tabId);
+ assert.equal(agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry', 'owned-retry').ok, true);
+ assert.equal(agent._failureRecovery.monitoringRetries.get(tabId).grantToken, retry.grantToken);
+ assert.equal(agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry', 'different-retry').ok, false);
+ assert.doesNotMatch(JSON.stringify(offer), /owned-document|browser-document|grantToken|taskSignature/);
+ assert.equal(agent._failureRecovery.takeMonitoringRetry(agent._failureRecoveryBinding(tabId), 'different-retry'), null);
+ assert.equal(agent._failureRecovery.takeMonitoringRetry(agent._failureRecoveryBinding(tabId), 'owned-retry'), retry);
+ assert.equal(agent._failureRecovery.takeMonitoringRetry(agent._failureRecoveryBinding(tabId), 'owned-retry'), null);
+ assert.equal(agent._pendingFailureRecoveryForUi(tabId), null);
+ });
+
+ test(`${browser}: rejected startup restores the monitoring Retry option with fresh UI authority`, async () => {
+ const agent = setup(), offer = monitoringFixture(agent), updates = [];
+ agent._finishPageFeedbackRun(tabId);
+ agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry', 'failed-start');
+ agent.setRunStartGuard(() => { throw new Error('fixture startup denied'); });
+ await assert.rejects(agent._claimRunEntry(tabId, 'interactive', {
+ trustedContinuation: true, detachedRequestId: 'failed-start' }, (type, data) => updates.push({ type, data })), /startup denied/);
+ const restored = agent._pendingFailureRecoveryForUi(tabId);
+ assert.equal(agent.isRunning(tabId), false);
+ assert.equal(restored.retryMode, 'action_monitoring_relaxed');
+ assert.notEqual(restored.recoveryId, offer.recoveryId);
+ assert.equal(updates.filter(update => update.type === 'failure_recovery').length, 1);
+ assert.equal(agent.requestFailureRecovery(tabId, restored.recoveryId, 'retry', 'another-start').ok, true);
+ });
+
+ test(`${browser}: monitoring grant expires on intervention, navigation and exact task changes`, () => {
+ for (const change of ['input', 'navigation', 'task', 'document']) {
+ const agent = setup(), offer = monitoringFixture(agent);
+ agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry', 'retry-request');
+ const retry = agent._failureRecovery.takeMonitoringRetry(agent._failureRecoveryBinding(tabId), 'retry-request');
+ const run = agent._pageFeedbackRuns.get(tabId);
+ run.monitoringRetry = retry; retry.started = true;
+ agent._runningTabs.add(tabId);
+ agent._beginSteeringRun(tabId, () => {}, { detachedRequestId: 'retry-request' });
+ assert.equal(agent._monitoringRetryCurrent(tabId), retry);
+ agent._queuePageFeedback(tabId, { kind: 'dom', source: 'unknown', frameId: 0, target: 'span' });
+ assert.equal(agent._monitoringRetryCurrent(tabId), retry, 'Unowned DOM-only counters are tolerated in the approved retry.');
+ if (change === 'input') agent._queuePageFeedback(tabId, { kind: 'input', source: 'user', frameId: 0 });
+ if (change === 'navigation') agent.observePageNavigation({ tabId, frameId: 0, url: run.url }, 'history');
+ if (change === 'task') agent.conversations.get(tabId).push({ role: 'user', content: 'A changed exact task.' });
+ if (change === 'document') run.frames.get(0).token = 'replacement-document';
+ assert.equal(agent._monitoringRetryCurrent(tabId), null, change);
+ assert.equal(retry.invalidated, true, change);
+ agent._releaseRunEntry(tabId);
+ assert.equal(agent._pendingFailureRecoveryForUi(tabId), null);
+ }
+ });
+
+ test(`${browser}: accepted startup correction permanently expires a retry even if startup subsequently rejects`, async () => {
+ const agent = setup(), offer = monitoringFixture(agent);
+ agent._finishPageFeedbackRun(tabId);
+ agent.requestFailureRecovery(tabId, offer.recoveryId, 'retry', 'corrected-start');
+ let rejectGuard, entered;
+ const ready = new Promise(resolve => { entered = resolve; });
+ agent.setRunStartGuard(() => { entered(); return new Promise((_resolve, reject) => { rejectGuard = reject; }); });
+ const attempt = agent._claimRunEntry(tabId, 'interactive', { trustedContinuation: true, detachedRequestId: 'corrected-start' }, () => {});
+ await ready;
+ assert.equal(agent.steerMessage(tabId, 'Cancel this task.', { requestId: 'corrected-start', messageId: 'correction' }).accepted, true);
+ rejectGuard(new Error('startup rejected after correction'));
+ await assert.rejects(attempt, /startup rejected after correction/);
+ assert.equal(agent._pendingFailureRecoveryForUi(tabId), null);
+ assert.equal(agent._monitoringRetryStarts.has(tabId), false);
+ });
+
+ test(`${browser}: grant metadata is finite and its token stays private`, () => {
+ const metadata = projectTraceEventData('note', { note: 'page_action_validation', extra: {
+ retryMode: 'action_monitoring_relaxed', certificationFailure: 'owner_unresolved', predicate: 'target_changed',
+ monitoringRelaxed: true, targetCount: 30, captureMs: 100, grantToken: 'private-token', text: 'private-body' } });
+ assert.equal(metadata.extra.retryMode, 'action_monitoring_relaxed');
+ assert.equal(metadata.extra.certificationFailure, 'owner_unresolved');
+ assert.equal(metadata.extra.monitoringRelaxed, true);
+ assert.doesNotMatch(JSON.stringify(metadata), /private-token|private-body/);
+ const bad = projectTraceEventData('note', { note: 'page_action_validation', extra: {
+ certificationFailure: 'private-body', predicate: 'private-secret', monitoringRelaxed: 'private-mode', captureMs: 'private-time' } });
+ assert.doesNotMatch(JSON.stringify(bad), /private-body|private-secret|private-mode|private-time/);
+ });
+
+ test(`${browser}: preparation-only passive veto retains the context failure after original-target revalidation`, async () => {
+ const agent = setup(); monitoringFixture(agent);
+ const run = agent._pageFeedbackRuns.get(tabId);
+ const state = run.modelState = { token: run.token, url: run.url, frames: new Map(run.frames), steeringRevision: 0,
+ actionBinding: { runToken: run.token, documentToken: 'owned-document', snapshotToken: 'original-observation' } };
+ const previous = extensionApi.tabs.sendMessage;
+ let changed = false, validations = 0;
+ extensionApi.tabs.sendMessage = async (_tab, message) => {
+ assert.equal(message.action, 'page_monitor_validate_model'); validations++;
+ return { ready: false, runToken: run.token, documentToken: 'owned-document', snapshotToken: 'original-observation',
+ ...(changed ? { reason: 'target_changed' } : { uncertified: true, certificationFailure: 'owner_unresolved' }) };
+ };
+ try {
+ run.actionContextUnverified = true;
+ const honest = await agent._pageFeedbackPreparationFailure(tabId, 'click_ax', { ref_id: 'ref_original' }, state);
+ assert.equal(honest.noDispatch, true);
+ assert.equal(honest.errorCode, 'action_context_unverified');
+ assert.equal(honest.actionContextUnverified, true);
+ changed = true;
+ const drift = await agent._pageFeedbackPreparationFailure(tabId, 'click_ax', { ref_id: 'ref_original' }, state);
+ assert.equal(drift.actionContextUnverified, undefined, 'Real target changes cannot be reported as the bypassable certificate failure.');
+ assert.equal(validations, 2);
+ } finally { extensionApi.tabs.sendMessage = previous; agent._finishPageFeedbackRun(tabId); }
+ });
+
test(`${browser}: recovery registry expires private task signatures even when normalized task keys match`, () => {
const registry = new FailureRecoveryRegistry();
const binding = { tabId, conversationId: 'c', taskKey: 'tk_same', taskSignature: 'exact-first' };
diff --git a/test/page-feedback-progress.mjs b/test/page-feedback-progress.mjs
index e0572d90d..4acf085a4 100644
--- a/test/page-feedback-progress.mjs
+++ b/test/page-feedback-progress.mjs
@@ -244,13 +244,18 @@ for (const build of ['chrome', 'firefox']) {
const offer = updates.find(u => u.type === 'failure_recovery')?.data;
assert.equal(offer?.stage, 'action_preparation');
assert.equal(offer.code, condition === 'unsupported' ? 'action_context_unverified' : 'page_unstable');
- assert.equal(offer.retryMode, 'resume');
+ const retryMode = condition === 'unsupported' ? 'action_monitoring_relaxed' : 'resume';
+ assert.equal(offer.retryMode, retryMode);
assert.equal(offer.message, result);
assert.match(result, /choose Retry/);
assert.equal(agent._pendingFailureRecoveryForUi(tab).recoveryId, offer.recoveryId);
- assert.deepEqual(agent.requestFailureRecovery(tab, offer.recoveryId, streaming ? 'retry' : 'give_up'),
- { ok: true, continue: streaming, retryMode: 'resume' });
- assert.equal(agent._pendingFailureRecoveryForUi(tab), null);
+ assert.deepEqual(agent.requestFailureRecovery(tab, offer.recoveryId, streaming ? 'retry' : 'give_up', 'owned-recovery-' + tab),
+ { ok: true, continue: streaming, retryMode });
+ if (streaming && condition === 'unsupported') {
+ assert.equal(agent._pendingFailureRecoveryForUi(tab).recoveryId, offer.recoveryId,
+ 'Keep recovery until its reserved continuation actually claims the run.');
+ agent._failureRecovery.clear(tab);
+ } else assert.equal(agent._pendingFailureRecoveryForUi(tab), null);
}
assert.equal(agent._pageFeedbackRuns.size, 0);
} finally { api.tabs.sendMessage = oldSend; documents.delete(tab); }
diff --git a/test/page-monitor-passive-progress.mjs b/test/page-monitor-passive-progress.mjs
index c1dfa47d8..de4beacfe 100644
--- a/test/page-monitor-passive-progress.mjs
+++ b/test/page-monitor-passive-progress.mjs
@@ -147,7 +147,9 @@ async function modelMessage(page, action, params = {}) {
}
function assertOpaqueModelMetadata(result) {
- const allowed = new Set(['ready', 'runToken', 'documentToken', 'snapshotToken', 'targetCount', 'revision', 'code', 'reason', 'uncertified', 'focusedTargetAvailable']);
+ const allowed = new Set(['ready', 'runToken', 'documentToken', 'snapshotToken', 'targetCount', 'revision', 'code', 'reason', 'uncertified', 'focusedTargetAvailable', 'certificationFailure', 'monitoringRelaxed']);
+ if (result.certificationFailure) assert.ok(['owner_unresolved', 'identity_unobserved', 'capture_limit'].includes(result.certificationFailure),
+ 'Certificate diagnostics contain only an enumerated failure predicate');
assert.ok(Object.keys(result).every(key => allowed.has(key)), 'Only opaque ownership metadata and counts may cross the content boundary');
assert.ok(Object.values(result).every(value => value === null || ['string', 'number', 'boolean'].includes(typeof value)),
'No node footprint, control values, file objects, or DOM arrays may be serialized');
diff --git a/test/page-monitor-relaxed-retry.mjs b/test/page-monitor-relaxed-retry.mjs
new file mode 100644
index 000000000..e3e4b9b26
--- /dev/null
+++ b/test/page-monitor-relaxed-retry.mjs
@@ -0,0 +1,321 @@
+import assert from 'node:assert/strict';
+import { readFile } from 'node:fs/promises';
+import { test } from 'node:test';
+import { chromium, firefox } from 'playwright';
+import { createNativeActionMarker } from '../firefox-companion/session.mjs';
+
+const token = 'approved-retry-private-grant-token';
+const runToken = 'approved-retry-run';
+const sources = new Map(await Promise.all(['chrome', 'firefox'].map(async build => [build,
+ await Promise.all(['rich-text-toolbar-heuristic.js', 'accessibility-tree.js', 'page-monitor.js', 'content.js']
+ .map(file => readFile(new URL(`../src/${build}/src/content/${file}`, import.meta.url), 'utf8')))])));
+
+function html(shape = 'generic') {
+ const editor = shape === 'generic' ? '<input id="editor" aria-label="Draft" value="Original draft">'
+ : `<div id="editor" contenteditable="true" role="textbox" aria-label="Draft" ${shape === 'x' ? 'data-testid="tweetTextarea_0"' : 'class="ProseMirror"'}><p>Original draft</p></div>`;
+ const button = shape === 'bluesky' ? '<div id="send" role="button" tabindex="0" data-testid="composerPublishBtn" aria-label="Publish">Post</div>'
+ : `<button id="send" type="button" ${shape === 'x' ? 'data-testid="tweetButtonInline"' : ''}>Send</button>`;
+ return `<!doctype html><style>body{margin:0;font:16px sans-serif;height:1500px}
+ #composer{margin:25px;width:450px;padding:10px}#editor{width:320px;min-height:55px;border:1px solid}
+ #send{display:block;width:100px;height:35px;margin-top:12px}#counter{height:24px}#outside{position:absolute;left:650px;top:20px}
+ #human{position:absolute;left:650px;top:120px}</style><div id="composer" ${shape === 'bluesky' ? 'role="dialog" aria-modal="true"' : ''}>
+ <form id="form" action="/original-destination" method="post">${editor}${button}<span id="counter">0</span></form></div>
+ <aside id="outside">0</aside><button id="human">Human control</button>`;
+}
+
+async function fixture(browser, build, { shape = 'generic', granted = true, grantDocument = true } = {}) {
+ const context = await browser.newContext({ viewport: { width: 1000, height: 750 } });
+ await context.route('https://relaxed-monitor.invalid/**', route => route.fulfill({ contentType: 'text/html', body: html(shape) }));
+ await context.addInitScript(({ granted, grantDocument, token, runToken }) => {
+ window.qaGranted = false; window.qaRunToken = runToken; window.qaFeedback = []; window.qaClicks = 0;
+ const listeners = [];
+ window.chrome = window.browser = { runtime: { getURL: value => value,
+ onMessage: { addListener: fn => listeners.push(fn), removeListener() {} },
+ async sendMessage(message) {
+ if (message.action === 'get_page_monitor_state') return { active: true, runToken: qaRunToken,
+ documentToken: message.documentToken, ...(qaGranted ? { relaxedMonitorGrant: {
+ grantToken: token, runToken: qaRunToken, documentToken: grantDocument ? message.documentToken : 'another-document',
+ } } : {}) };
+ if (message.action === 'page_feedback') qaFeedback.push(message.feedback);
+ return { accepted: true };
+ },
+ } };
+ window.qaMessage = (action, params = {}) => new Promise((resolve, reject) => {
+ const timeout = setTimeout(() => reject(new Error(`No response for ${action}`)), 5000);
+ let answered = false;
+ const respond = value => { answered = true; clearTimeout(timeout); resolve(value); };
+ for (const listener of listeners) {
+ const result = listener(action === 'page_monitor_state'
+ ? { target: 'content', action, runToken: qaRunToken, ...params }
+ : { target: 'content', action, params: { runToken: qaRunToken, ...params } }, {}, respond);
+ if (answered || result === true) return;
+ }
+ clearTimeout(timeout); resolve(undefined);
+ });
+ }, { granted, grantDocument, token, runToken });
+ for (const content of sources.get(build)) await context.addInitScript({ content });
+ const page = await context.newPage();
+ await page.goto('https://relaxed-monitor.invalid/start');
+ await page.waitForFunction(() => __wbPageMonitor?.active);
+ await page.waitForTimeout(180);
+ await page.evaluate(async granted => { qaGranted = granted; await qaMessage('page_monitor_state', { active: true }); }, granted);
+ await page.evaluate(() => document.getElementById('send').addEventListener('click', () => qaClicks++));
+ return { page, context };
+}
+
+async function capture(page, selector = '#send', extra = {}) {
+ return page.evaluate(({ selector, extra, token }) => qaMessage('page_monitor_capture_model', {
+ includeTree: true, relaxedMonitorGrantToken: token, actionTarget: { tool: 'click_ax', selector }, ...extra,
+ }), { selector, extra, token });
+}
+async function validate(page, snapshot, selector = '#send', tool = 'click_ax', extra = {}) {
+ return page.evaluate(({ snapshot, selector, tool, extra, token }) => qaMessage('page_monitor_validate_model', {
+ snapshotToken: snapshot.snapshotToken, selector, tool, relaxedMonitorGrantToken: token, ...extra,
+ }), { snapshot, selector, tool, extra, token });
+}
+async function prepare(page, snapshot, selector = '#send', tool = 'click_ax') {
+ return page.evaluate(({ snapshot, selector, tool, token }) => qaMessage('page_monitor_prepare', {
+ expectedModelSnapshot: snapshot.snapshotToken, selector, tool, relaxedMonitorGrantToken: token,
+ allowPassiveRebase: true, operationId: 'owned-retry-action',
+ }), { snapshot, selector, tool, token });
+}
+async function fence(page, kind = 'click') {
+ return page.evaluate(kind => qaMessage('page_monitor_dispatch', {
+ operationId: 'owned-retry-action', kind, fenceOnly: true,
+ }), kind);
+}
+
+for (const [build, engine] of [['chrome', chromium], ['firefox', firefox]]) {
+ test(`${build}: approved retry binds an observed original target through continual passive churn`, { timeout: 120000 }, async t => {
+ const browser = await engine.launch({ headless: true });
+ try {
+ for (const shape of ['generic', 'x', 'bluesky']) await t.test(shape, async () => {
+ const f = await fixture(browser, build, { shape });
+ try {
+ const snapshot = await capture(f.page);
+ assert.equal(snapshot.ready, true, JSON.stringify(snapshot));
+ assert.equal(JSON.stringify(snapshot).includes(token), false);
+ await f.page.evaluate(() => {
+ window.qaChurn = setInterval(() => {
+ counter.textContent = String(Number(counter.textContent) + 1);
+ outside.textContent = String(Number(outside.textContent) + 1);
+ }, 8);
+ });
+ await f.page.waitForTimeout(80);
+ const binding = await validate(f.page, snapshot);
+ assert.equal(binding.ready, true, JSON.stringify(binding));
+ assert.equal(binding.relaxed, true); assert.equal(binding.bindingKind, 'relaxed_target');
+ assert.equal(JSON.stringify(binding).includes('Original draft'), false);
+ assert.equal(JSON.stringify(binding).includes(token), false);
+ assert.equal((await prepare(f.page, snapshot)).modelBindingValid, true);
+ await f.page.waitForTimeout(30);
+ assert.equal((await fence(f.page)).ready, true);
+ const result = await f.page.evaluate(() => qaMessage('click_ax', { ref_id: __wb_ax_ref(send) }));
+ assert.equal(result.success, true, JSON.stringify(result));
+ assert.equal(await f.page.evaluate(() => qaClicks), 1);
+ } finally { await f.context.close(); }
+ });
+ } finally { await browser.close(); }
+ });
+
+ test(`${build}: no private state grant, mismatched grant or old observation preserves ordinary monitoring`, { timeout: 120000 }, async t => {
+ const browser = await engine.launch({ headless: true });
+ try {
+ for (const options of [{ granted: false }, { grantDocument: false }]) await t.test(JSON.stringify(options), async () => {
+ const f = await fixture(browser, build, options);
+ try {
+ const snapshot = await capture(f.page);
+ const binding = await validate(f.page, snapshot);
+ assert.equal(binding.ready, false); assert.equal(binding.relaxed, undefined);
+ assert.equal(binding.certificationFailure, 'owner_unresolved');
+ assert.equal((await prepare(f.page, snapshot)).modelBindingValid, false);
+ assert.equal((await fence(f.page)).ready, false);
+ await f.page.evaluate(({ token, runToken }) => qaMessage('page_monitor_state', { active: true,
+ relaxedMonitorGrant: { grantToken: token, runToken, documentToken: 'forged' } }), { token, runToken });
+ assert.equal((await validate(f.page, snapshot)).relaxed, undefined);
+ } finally { await f.context.close(); }
+ });
+ await t.test('grant cannot bless a snapshot captured without authority or a real observed tree', async () => {
+ const f = await fixture(browser, build, { granted: false });
+ try {
+ const original = await capture(f.page);
+ await f.page.evaluate(async () => { qaGranted = true; await qaMessage('page_monitor_state', { active: true }); });
+ assert.equal((await validate(f.page, original)).relaxed, undefined);
+ const metadata = await capture(f.page, '#send', { includeTree: false });
+ assert.equal((await validate(f.page, metadata)).relaxed, undefined);
+ const fresh = await capture(f.page);
+ assert.equal((await validate(f.page, fresh)).relaxed, true);
+ assert.equal((await validate(f.page, fresh, '#send', 'click_ax', { relaxedMonitorGrantToken: 'arbitrary-model-token' })).ready, false);
+ } finally { await f.context.close(); }
+ });
+ } finally { await browser.close(); }
+ });
+
+ test(`${build}: target changes, lost actionability, human input and navigation expire the retry`, { timeout: 180000 }, async t => {
+ const browser = await engine.launch({ headless: true });
+ try {
+ for (const [name, mutate] of [
+ ['same-markup replacement', page => page.evaluate(() => send.replaceWith(send.cloneNode(true)))],
+ ['same node moved', page => page.evaluate(() => composer.append(send))],
+ ['move then restore', page => page.evaluate(() => { composer.append(send); form.append(send); })],
+ ['ancestor moved', page => page.evaluate(() => { const wrapper = document.createElement('div'); document.body.append(wrapper); wrapper.append(composer); })],
+ ['form changed', page => page.evaluate(() => form.action = '/different-destination')],
+ ['form change then restore', page => page.evaluate(() => { form.action = '/different-destination'; form.setAttribute('action', '/original-destination'); })],
+ ['target label changed', page => page.evaluate(() => send.textContent = 'Different action')],
+ ['disabled target', page => page.evaluate(() => send.disabled = true)],
+ ['hidden target', page => page.evaluate(() => send.style.display = 'none')],
+ ['covering overlay', page => page.evaluate(() => { const overlay = document.createElement('div'); overlay.style.cssText = 'position:fixed;inset:0;background:white;z-index:1000'; document.body.append(overlay); })],
+ ['human pointer', page => page.click('#human')],
+ ['human key', page => page.keyboard.press('Escape')],
+ ['unowned script focus', page => page.evaluate(() => human.focus())],
+ ['same-document navigation', page => page.evaluate(() => history.pushState({}, '', '/another-page'))],
+ ['Stop', page => page.evaluate(() => qaMessage('page_monitor_state', { active: false }))],
+ ]) await t.test(name, async () => {
+ const f = await fixture(browser, build);
+ try {
+ const snapshot = await capture(f.page);
+ assert.equal((await prepare(f.page, snapshot)).modelBindingValid, true);
+ await mutate(f.page);
+ assert.equal((await fence(f.page)).ready, false, name);
+ assert.equal((await validate(f.page, snapshot)).ready, false, name);
+ assert.equal(await f.page.evaluate(() => qaClicks), 0);
+ } finally { await f.context.close(); }
+ });
+ await t.test('navigation to a new document cannot reuse the old grant or snapshot', async () => {
+ const f = await fixture(browser, build);
+ try {
+ const snapshot = await capture(f.page);
+ await f.page.reload(); await f.page.waitForFunction(() => __wbPageMonitor?.active);
+ assert.equal((await validate(f.page, snapshot)).ready, false);
+ } finally { await f.context.close(); }
+ });
+ await t.test('a dispatch cannot exchange the approved selector for a different original node', async () => {
+ const f = await fixture(browser, build);
+ try {
+ const snapshot = await capture(f.page);
+ assert.equal((await prepare(f.page, snapshot)).modelBindingValid, true);
+ const result = await f.page.evaluate(() => qaMessage('page_monitor_dispatch', {
+ operationId: 'owned-retry-action', selector: '#human', kind: 'click', fenceOnly: true,
+ }));
+ assert.equal(result.ready, false); assert.equal(result.preparationVeto.predicate, 'target_changed');
+ } finally { await f.context.close(); }
+ });
+ for (const [name, mutate] of [
+ ['text changed and restored', () => { send.textContent = 'Delete everything'; send.textContent = 'Send'; }],
+ ['form changed and restored', () => { form.action = '/different'; form.setAttribute('action', '/original-destination'); }],
+ ['parent changed and restored', () => { composer.append(send); form.append(send); }],
+ ]) await t.test(`inference tombstone before first validate: ${name}`, async () => {
+ const f = await fixture(browser, build);
+ try {
+ const snapshot = await capture(f.page);
+ await f.page.evaluate(mutate);
+ const validation = await validate(f.page, snapshot);
+ assert.equal(validation.ready, false); assert.equal(validation.reason, 'target_changed');
+ } finally { await f.context.close(); }
+ });
+ } finally { await browser.close(); }
+ });
+
+ test(`${build}: owned focus, scroll and repeated native input keep the original relaxed target bound`, { timeout: 120000 }, async t => {
+ const browser = await engine.launch({ headless: true });
+ try {
+ for (const shape of ['generic', 'x', 'bluesky']) await t.test(shape, async () => {
+ const f = await fixture(browser, build, { shape });
+ try {
+ await f.page.evaluate(() => composer.style.marginTop = '520px');
+ const snapshot = await capture(f.page, '#editor');
+ assert.equal((await validate(f.page, snapshot, '#editor', 'type_ax')).relaxed, true);
+ assert.equal((await prepare(f.page, snapshot, '#editor', 'type_ax')).modelBindingValid, true);
+ await f.page.evaluate(() => {
+ const finish = __wbPageMonitor.beginContentAction('type_ax', { ref_id: __wb_ax_ref(editor), _bidiPrepare: true });
+ __wbPageMonitor.withLocalPreparation('scroll', editor, () => editor.scrollIntoView({ block: 'center' }));
+ __wbPageMonitor.withLocalPreparation('focus', editor, () => editor.focus());
+ finish();
+ });
+ assert.ok(await f.page.evaluate(() => scrollY > 0), 'Owned preparation really scrolled the document');
+ await f.page.waitForTimeout(70);
+ let receipt = await fence(f.page, 'input'); assert.equal(receipt.ready, true, JSON.stringify(receipt));
+ for (let sequence = 1; sequence <= 2; sequence++) {
+ await f.page.evaluate(() => { counter.textContent += '1'; outside.textContent += '1'; });
+ const validation = await f.page.evaluate(guard => qaMessage('page_monitor_validate', { ...guard, kind: 'input' }), receipt.guard);
+ assert.equal(validation.ready, true, JSON.stringify(validation));
+ receipt.guard.revision = validation.revision;
+ const marker = createNativeActionMarker(receipt.guard, 'input', sequence, false);
+ await f.page.evaluate(marker => editor.setAttribute('data-webbrain-native-action', marker), marker);
+ await f.page.evaluate(() => { counter.textContent += '2'; outside.textContent += '2'; });
+ await f.page.keyboard.insertText(String(sequence));
+ assert.equal(await f.page.evaluate(() => editor.hasAttribute('data-webbrain-native-action')), false);
+ }
+ assert.match(await f.page.evaluate(() => editor.value ?? editor.textContent), /12/);
+ assert.equal(await f.page.evaluate(() => qaFeedback.some(event => event.source === 'user')), false);
+ await f.page.evaluate(() => qaMessage('page_monitor_finish', { operationId: 'owned-retry-action' }));
+ await f.page.waitForTimeout(210);
+ const afterInput = await capture(f.page);
+ assert.equal(afterInput.monitoringRelaxed, true);
+ assert.equal((await validate(f.page, afterInput)).relaxed, true);
+ await f.page.evaluate(() => form.action = '/changed-after-input');
+ assert.equal((await f.page.evaluate(guard => qaMessage('page_monitor_validate', { ...guard, kind: 'input' }), receipt.guard)).ready, false);
+ } finally { await f.context.close(); }
+ });
+ } finally { await browser.close(); }
+ });
+
+ test(`${build}: a completed owned editor write preserves the retry for a fresh next-button observation`, { timeout: 60000 }, async t => {
+ const browser = await engine.launch({ headless: true });
+ try {
+ for (const shape of ['generic', 'x']) await t.test(shape, async () => {
+ const f = await fixture(browser, build, { shape });
+ try {
+ const snapshot = await capture(f.page, '#editor');
+ assert.equal(snapshot.monitoringRelaxed, true);
+ assert.equal((await prepare(f.page, snapshot, '#editor', 'set_field')).modelBindingValid, true);
+ const input = await f.page.evaluate(() => qaMessage('set_field', { ref_id: __wb_ax_ref(editor), text: 'Exact completed owned draft' }));
+ assert.equal(input.success, true, JSON.stringify(input));
+ await f.page.waitForTimeout(220);
+ const next = await capture(f.page);
+ assert.equal(next.monitoringRelaxed, true, JSON.stringify(next));
+ assert.equal((await validate(f.page, next)).relaxed, true);
+ assert.equal((await prepare(f.page, next)).modelBindingValid, true);
+ const click = await f.page.evaluate(() => qaMessage('click_ax', { ref_id: __wb_ax_ref(send) }));
+ assert.equal(click.success, true, JSON.stringify(click));
+ assert.equal(await f.page.evaluate(() => qaClicks), 1);
+ assert.equal(await f.page.evaluate(() => editor.value ?? editor.textContent), 'Exact completed owned draft');
+ } finally { await f.context.close(); }
+ });
+ } finally { await browser.close(); }
+ });
+ if (build === 'chrome') test(`${build}: an exact private trusted-typing probe keeps the original relaxed preparation certificate`, { timeout: 60000 }, async () => {
+ const browser = await engine.launch({ headless: true });
+ const f = await fixture(browser, build, { shape: 'bluesky' });
+ try {
+ const snapshot = await capture(f.page, '#editor');
+ assert.equal((await prepare(f.page, snapshot, '#editor', 'set_field')).modelBindingValid, true);
+ const response = await f.page.evaluate(() => qaMessage('set_field', { ref_id: __wb_ax_ref(editor), text: 'Owned managed input' }));
+ assert.equal(response.trustedTypeRequired, true); assert.equal(response.noDispatch, true);
+ const prepared = await f.page.evaluate(({ snapshot, token }) => qaMessage('ax_prepare_field_for_trusted_type', {
+ ref_id: __wb_ax_ref(editor), expectedModelSnapshot: snapshot.snapshotToken, relaxedMonitorGrantToken: token, allowPassiveRebase: true,
+ }), { snapshot, token });
+ assert.equal(prepared.success, true, JSON.stringify(prepared));
+ const current = await capture(f.page, '#editor');
+ assert.equal(current.monitoringRelaxed, true);
+ assert.equal((await validate(f.page, snapshot, '#editor', 'set_field')).relaxed, true);
+ } finally { await f.context.close(); await browser.close(); }
+ });
+ test(`${build}: completed owned menu activation keeps the run while its old snapshot expires`, { timeout: 60000 }, async () => {
+ const browser = await engine.launch({ headless: true });
+ const f = await fixture(browser, build);
+ try {
+ const snapshot = await capture(f.page);
+ assert.equal((await prepare(f.page, snapshot)).modelBindingValid, true);
+ await f.page.evaluate(() => send.addEventListener('click', () => {
+ send.setAttribute('aria-expanded', 'true'); send.textContent = 'Close options';
+ }));
+ assert.equal((await f.page.evaluate(() => qaMessage('click_ax', { ref_id: __wb_ax_ref(send) }))).success, true);
+ await f.page.waitForTimeout(210);
+ const fresh = await capture(f.page, '#human');
+ assert.equal(fresh.monitoringRelaxed, true);
+ assert.equal((await validate(f.page, fresh, '#human')).relaxed, true);
+ assert.equal((await validate(f.page, snapshot)).ready, false, 'Completion never rewrites the old inference certificate');
+ } finally { await f.context.close(); await browser.close(); }
+ });
+}
diff --git a/test/quality/relaxed-monitoring-retry-mv3.mjs b/test/quality/relaxed-monitoring-retry-mv3.mjs
new file mode 100644
index 000000000..37345eab8
--- /dev/null
+++ b/test/quality/relaxed-monitoring-retry-mv3.mjs
@@ -0,0 +1,417 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs/promises';
+import http from 'node:http';
+import os from 'node:os';
+import path from 'node:path';
+import { chromium } from 'playwright';
+import { build as bundle } from 'esbuild';
+
+// Installed MV3 regression with a separately labelled scripted provider. The
+// real background, Agent, monitor, side-panel Retry, native input and completion
+// gates run unchanged. These owned social-shaped shells exercise unsupported
+// generic controls. Actual ProseMirror model/view owns the managed editors;
+// the full native social composer contracts are covered by their own suites.
+const providerLabel = 'deterministic scripted provider; installed production runtime';
+const out = path.resolve(process.env.WEBBRAIN_QUALITY_ARTIFACTS
+ || 'ci/artifacts/quality-audit-2026-10-13/relaxed-monitoring-retry-mv3');
+await fs.mkdir(out, { recursive: true });
+const body = 'Owned retry draft: exact spaces and Turkish ıİşŞğĞüÜöÖçÇ 🧠.';
+const editorBundle = (await bundle({ entryPoints: [path.resolve('test/fixtures/bluesky-composer-editor.mjs')],
+ bundle: true, write: false, format: 'iife', globalName: 'qaEditorLibrary', platform: 'browser' })).outputFiles[0].text;
+const cases = [
+ { name: 'generic_editor_menu', url: 'https://relaxed-monitoring.webbrain.invalid/editor', editorName: 'Owned draft', menuName: 'Formatting options' },
+ { name: 'generic_preparation_editor_menu', url: 'https://relaxed-monitoring.webbrain.invalid/preparation-editor', editorName: 'Owned draft', menuName: 'Formatting options', preparationOnly: true },
+ { name: 'generic_managed_editor_menu', url: 'https://relaxed-monitoring.webbrain.invalid/managed-editor', editorName: 'Owned draft', menuName: 'Formatting options', managed: true },
+ { name: 'x_shaped_draft', url: 'https://x.com/home', editorName: 'Post text', menuName: 'Composer options', managed: true },
+ { name: 'bluesky_shaped_draft', url: 'https://bsky.app/', editorName: "What's up?", menuName: 'Draft options', managed: true },
+].filter(item => !process.env.WEBBRAIN_QUALITY_RELAXED_CASE || item.name === process.env.WEBBRAIN_QUALITY_RELAXED_CASE);
+assert.ok(cases.length, 'Select an existing relaxed-monitoring Retry case.');
+
+const textOf = content => typeof content === 'string' ? content : Array.isArray(content)
+ ? content.filter(part => part?.type === 'text').map(part => part.text || '').join('\n') : '';
+function toolObject(content) {
+ const text = textOf(content);
+ // Preserve the production nonce-bound untrusted-page envelope.
+ for (let start = text.indexOf('{'); start >= 0; start = text.indexOf('{', start + 1)) {
+ let depth = 0, quoted = false, escaped = false;
+ for (let cursor = start; cursor < text.length; cursor++) {
+ const char = text[cursor];
+ if (quoted) { if (escaped) escaped = false; else if (char === '\\') escaped = true; else if (char === '"') quoted = false; }
+ else if (char === '"') quoted = true;
+ else if (char === '{') depth++;
+ else if (char === '}' && --depth === 0) {
+ try { return JSON.parse(text.slice(start, cursor + 1)); } catch { break; }
+ }
+ }
+ }
+ return null;
+}
+function latestTools(messages) {
+ const names = new Map();
+ for (const message of messages || []) for (const call of message.tool_calls || []) names.set(call.id, call.function?.name);
+ return (messages || []).filter(message => message.role === 'tool')
+ .map(message => ({ name: names.get(message.tool_call_id) || message.name, result: toolObject(message.content) }));
+}
+function intent(current) {
+ return { request_kind: 'execute', scope_relation: 'new', requires_state_change: true,
+ requires_submission: false, messaging: null,
+ deliverables: ['The exact owned draft is present once and the local options menu is open.'],
+ allows_planner_shaped_result: false, allows_app_state_tool_evidence: false,
+ completion_requirements: { download: false }, read_scope: 'visible_page',
+ summary: 'Write the exact owned draft, open its local options once, freshly verify both and leave it unsubmitted.',
+ steps: [{ id: '1', action: `Write ${current.editorName}, observe ${current.menuName}, open its options and verify the draft.` }],
+ memory: { use_progress_ledger: false, progress_action: null }, scheduling: null, risks: [] };
+}
+function fixtureHtml(specification) {
+ const social = ['x_shaped_draft', 'bluesky_shaped_draft'].includes(specification.name);
+ const shell = specification.name === 'x_shaped_draft'
+ ? '<div class="tabs"><button type="button" role="tab" aria-selected="true">For you</button><button type="button" role="tab" aria-selected="false">Following</button></div>'
+ : specification.name === 'bluesky_shaped_draft'
+ ? '<div class="tabs"><button type="button" role="tab" aria-selected="true">Following</button><button type="button" role="tab" aria-selected="false">Discover</button></div>' : '';
+ // No entity/form/dialog/heading owner is supplied. The generic fallback is
+ // intentional, even when an app-like surrounding shell is present.
+ return `<!doctype html><meta charset="utf-8"><title>Owned ${specification.name} Retry fixture</title><style>
+body{margin:0;background:#fff;color:#111;font:17px system-ui}.layout{margin:22px;display:flex;gap:28px}
+.navigation{width:150px}.work-area{width:620px}.tabs{display:flex;gap:20px;height:62px}
+#composer{padding:16px;border:1px solid #bbb;width:570px}#editor{height:120px;padding:8px;border:1px solid #777;white-space:pre-wrap}#editor p{margin:0}
+.toolbar{display:flex;justify-content:space-between;margin-top:14px}button{font:inherit;padding:8px;cursor:pointer}
+aside{width:240px;padding:12px;border:1px solid #ddd}#feed-item{height:80px}#menu{padding:12px;margin-top:8px;border:1px solid #777}
+</style>${specification.managed ? '<script src="/owned-fixture/editor.js"></script>' : ''}<div class="layout">${social ? '<div class="navigation">Owned QA profile<br>Local intercepted timeline</div>' : ''}
+<div class="work-area">${shell}<div id="composer">${specification.managed ? '<div id="editor-host"></div><button id="publish" disabled hidden aria-disabled="true">Post</button>' : `<div id="editor" role="textbox" contenteditable="true" aria-label="${specification.editorName.replaceAll('"', '&quot;')}" aria-multiline="true"></div>`}
+<div class="toolbar"><button id="options" type="button">${specification.menuName}</button>${social ? '<button type="button" disabled>Post</button>' : ''}</div>
+<div id="menu" hidden><span>Owned local formatting menu is open</span></div><div id="draft-result" role="status"></div></div></div>
+<aside><span id="counter">Passive feed revision 0</span><div id="feed-item">Unrelated local feed update 0</div></aside></div><script>
+window.qaEvents=[];window.qaInputCount=0;window.qaOptionsClicks=0;window.qaTicks=0;
+${specification.managed ? `qaEditorLibrary.mountEditor('');qaEditorView.dom.id='editor';qaEditorView.dom.setAttribute('aria-label',${JSON.stringify(specification.editorName)});` : ''}
+window.qaOriginalEditor=document.querySelector('#editor');window.qaOriginalOptions=document.querySelector('#options');
+window.qaPassiveTick=()=>{qaTicks++;document.querySelector('#counter').textContent='Passive feed revision '+qaTicks;
+ document.querySelector('#feed-item').textContent='Unrelated local feed update '+qaTicks;};
+// The preparation case intentionally outruns the DOM debounce: local pending
+// changes must preserve the original unsupported-context diagnosis too.
+setInterval(qaPassiveTick,${specification.preparationOnly ? 80 : 200});
+for(const type of ['beforeinput','input','paste'])qaOriginalEditor.addEventListener(type,event=>{
+ qaEvents.push({type,trusted:event.isTrusted,inputType:event.inputType,data:event.data});
+ if(type==='input')qaInputCount++;
+},true);
+qaOriginalOptions.addEventListener('click',event=>{
+ qaOptionsClicks++;qaEvents.push({type:'options-click',trusted:event.isTrusted});
+ document.querySelector('#menu').hidden=false;
+ document.querySelector('#draft-result').textContent='Owned draft verified: '+qaOriginalEditor.innerText;
+});
+window.qaSnapshot=()=>({draft:${specification.managed ? 'qaSnapshotEditor().model' : 'qaOriginalEditor.innerText'},editorSame:qaOriginalEditor===document.querySelector('#editor'),
+ optionsSame:qaOriginalOptions===document.querySelector('#options'),optionsClicks:qaOptionsClicks,inputCount:qaInputCount,
+ menuOpen:!document.querySelector('#menu').hidden,ticks:qaTicks,events:qaEvents});
+</script>`;
+}
+
+let current, context, worker, harness, panel;
+const summaries = [], blockedRemote = [], auxiliaryRequests = [];
+const server = http.createServer(async (request, response) => {
+ try {
+ if (request.url !== '/v1/chat/completions') { response.writeHead(404).end(); return; }
+ let raw = ''; for await (const part of request) raw += part;
+ const input = JSON.parse(raw), system = textOf(input.messages?.[0]?.content);
+ let completion;
+ if (!input.tools?.length && input.messages?.length === 1 && input.messages[0].role === 'user' && system === 'Hi') {
+ // The installed settings/panel tests the configured provider normally.
+ auxiliaryRequests.push(input);
+ completion = { role: 'assistant', content: 'Owned scripted provider ready.' };
+ } else if (system.startsWith('Extract stable user preferences for WebBrain memory.')) {
+ auxiliaryRequests.push(input);
+ completion = { role: 'assistant', content: '{"memories":[]}' };
+ } else {
+ if (!current || !JSON.stringify(input.messages).includes(current.marker)) {
+ auxiliaryRequests.push(input);
+ assert.fail(`Every action provider request must retain the authentic owned task; unexpected subsystem: ${system.slice(0, 160)}`);
+ }
+ current.requests.push({ phase: current.phase, input });
+ if (system.startsWith("Compile the user's social-publication intent across languages.")) {
+ current.compilers++;
+ completion = { role: 'assistant', content: JSON.stringify({ version: 1, status: 'none',
+ actions: [], requirements: null, prohibited: [], reason: 'The authentic owned task requests an unsubmitted draft and local options only.' }) };
+ } else if (system.startsWith('Evaluate only the observable requirements of the genuine user task')) {
+ current.judges++;
+ const state = await current.page.evaluate(() => qaSnapshot());
+ assert.equal(state.draft, body); assert.equal(state.menuOpen, true); assert.equal(state.optionsClicks, 1);
+ completion = { role: 'assistant', content: JSON.stringify({ outcome: 'succeeded',
+ reason: 'Fresh page evidence shows the exact unsubmitted owned draft and its opened local options.' }) };
+ } else if (!input.tools?.length) completion = { role: 'assistant', content: JSON.stringify(intent(current)) };
+ else {
+ const tools = latestTools(input.messages), last = tools.at(-1);
+ const writes = tools.filter(tool => tool.name === 'set_field' && tool.result?.success === true);
+ const clicks = tools.filter(tool => tool.name === 'click_ax' && (tool.result?.success === true || tool.result?.dispatched === true));
+ let call;
+ if (!writes.length) call = { name: 'set_field', args: { ref_id: current.refs.editor, text: body, clear: true } };
+ else if (!clicks.length && last?.name !== 'get_accessibility_tree') call = { name: 'get_accessibility_tree', args: { filter: 'all' } };
+ else if (!clicks.length) call = { name: 'click_ax', args: { ref_id: current.refs.options } };
+ else if (last?.name !== 'get_accessibility_tree') call = { name: 'get_accessibility_tree', args: { filter: 'all' } };
+ else {
+ assert.ok(last.result?.pageContent?.includes(body.replace(/\s+/g, ' ')), 'Completion requires fresh draft evidence from the normal reader.');
+ assert.ok(last.result.pageContent.includes('Owned local formatting menu is open'));
+ call = { name: 'done', args: { outcome: 'success', summary: 'The exact owned draft and its opened local options were freshly verified.' } };
+ }
+ if (current.phase === 'strict') {
+ current.strictDecisions++;
+ assert.ok(current.strictDecisions <= 5, 'Strict unsupported context must stop after its five normal rejections.');
+ assert.equal(call.name, 'set_field');
+ } else current.retryDecisions++;
+ assert.ok(current.calls.length < 20, 'An owned Retry must make bounded progress.');
+ // Real page activity occurs after production model capture, throughout
+ // inference and native preparation. The original target stays mounted.
+ const beforeTicks = await current.page.evaluate(() => { qaPassiveTick(); return qaTicks; });
+ await new Promise(resolve => setTimeout(resolve, 360));
+ const afterTicks = await current.page.evaluate(() => qaTicks);
+ assert.ok(afterTicks > beforeTicks, 'Passive churn must continue while every decision is in flight.');
+ current.calls.push({ phase: current.phase, ...call, beforeTicks, afterTicks });
+ completion = { role: 'assistant', content: '', tool_calls: [{ id: `owned-relaxed-${current.name}-${current.calls.length}`,
+ type: 'function', function: { name: call.name, arguments: JSON.stringify(call.args) } }] };
+ }
+ }
+ response.writeHead(200, { 'Content-Type': 'application/json' }).end(JSON.stringify({ id: 'owned-relaxed-scripted-provider',
+ choices: [{ message: completion, finish_reason: completion.tool_calls ? 'tool_calls' : 'stop' }],
+ usage: { prompt_tokens: 100, completion_tokens: 30 } }));
+ } catch (error) {
+ if (current) current.providerError = error.stack;
+ response.writeHead(500, { 'Content-Type': 'application/json' }).end(JSON.stringify({ error: error.message }));
+ }
+});
+await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
+const profile = await fs.mkdtemp(path.join(os.tmpdir(), 'webbrain-relaxed-monitoring-retry-mv3-'));
+const evidenceReplacer = (key, value) => ['relaxedMonitorGrantToken', 'grantToken'].includes(key) && value
+ ? '[private grant redacted]' : typeof value === 'string' && value.startsWith('data:image/') ? '[owned screenshot omitted]' : value;
+
+try {
+ const extension = path.join(profile, 'extension');
+ await fs.cp(path.resolve('src/chrome'), extension, { recursive: true });
+ // Read-only diagnostics in a disposable extension clone. No Agent method or
+ // production decision is replaced, including recovery and completion.
+ await fs.appendFile(path.join(extension, 'src/background.js'), '\nglobalThis.relaxedFixtureAgent = agent;\n');
+ await fs.appendFile(path.join(extension, 'src/ui/sidepanel.js'),
+ '\nglobalThis.relaxedFixturePanelState = () => ({ currentTabId, renderedTabId, isProcessing });\n');
+ context = await chromium.launchPersistentContext(profile, { headless: true, channel: 'chromium', viewport: { width: 1280, height: 1000 },
+ args: [`--disable-extensions-except=${extension}`, `--load-extension=${extension}`,
+ '--host-resolver-rules=MAP * ~NOTFOUND, EXCLUDE 127.0.0.1'] });
+ await context.route('**/*', async route => {
+ const request = route.request(), requested = new URL(request.url());
+ if (requested.hostname === '127.0.0.1' && requested.port === String(server.address().port)) return route.continue();
+ const specification = cases.find(item => requested.href === item.url);
+ if (specification && request.isNavigationRequest() && request.method() === 'GET')
+ return route.fulfill({ contentType: 'text/html', body: fixtureHtml(specification) });
+ if (cases.some(item => new URL(item.url).origin === requested.origin)
+ && requested.pathname === '/owned-fixture/editor.js' && request.method() === 'GET')
+ return route.fulfill({ contentType: 'text/javascript', body: editorBundle });
+ if (/^https?:$/.test(requested.protocol)) {
+ blockedRemote.push({ method: request.method(), origin: requested.origin }); return route.abort('blockedbyclient');
+ }
+ return route.continue();
+ });
+ worker = context.serviceWorkers()[0] || await context.waitForEvent('serviceworker', { timeout: 30000 });
+ await worker.evaluate(() => {
+ const send = chrome.tabs.sendMessage.bind(chrome.tabs), command = chrome.debugger.sendCommand.bind(chrome.debugger), fetchOriginal = fetch.bind(globalThis);
+ globalThis.relaxedFixtureProbe = { transports: [], commands: [], monitorStateRequests: [], blocked: [] };
+ chrome.tabs.sendMessage = async (...args) => {
+ const result = await send(...args), message = args[1];
+ if (/page_monitor|set_field|type_ax|click_ax|ax_prepare|ax_resolve/.test(message.action)) relaxedFixtureProbe.transports.push({ tabId: args[0],
+ action: message.action, params: { tool: message.params?.tool, ref_id: message.params?.ref_id,
+ runToken: message.params?.runToken || message.runToken, snapshotToken: message.params?.snapshotToken,
+ expectedModelSnapshot: message.params?.expectedModelSnapshot,
+ relaxedMonitorGrantToken: message.params?.relaxedMonitorGrantToken, operationId: message.params?.operationId,
+ fenceOnly: message.params?.fenceOnly, allowPassiveRebase: message.params?.allowPassiveRebase }, result });
+ return result;
+ };
+ chrome.debugger.sendCommand = (...args) => {
+ relaxedFixtureProbe.commands.push({ method: args[1], params: args[2] }); return command(...args);
+ };
+ chrome.runtime.onMessage.addListener((message, sender) => {
+ if (message.action === 'get_page_monitor_state') relaxedFixtureProbe.monitorStateRequests.push({ tabId: sender.tab?.id,
+ documentToken: message.documentToken, frameId: sender.frameId });
+ });
+ globalThis.fetch = (input, options) => {
+ const target = new URL(typeof input === 'string' ? input : input.url);
+ if (target.hostname === '127.0.0.1' || ['chrome-extension:', 'data:', 'blob:'].includes(target.protocol)) return fetchOriginal(input, options);
+ relaxedFixtureProbe.blocked.push({ method: options?.method || 'GET', origin: target.origin });
+ return Promise.reject(new Error('The owned relaxed Retry fixture blocks external service requests.'));
+ };
+ });
+ harness = await context.newPage();
+ const extensionOrigin = `chrome-extension://${new URL(worker.url()).host}`;
+ await harness.goto(`${extensionOrigin}/src/ui/settings.html`);
+ const message = (action, data = {}) => harness.evaluate(({ action, data }) => chrome.runtime.sendMessage({ target: 'background', action, ...data }), { action, data });
+ await harness.evaluate(() => {
+ window.relaxedFixtureUpdates = [];
+ chrome.runtime.onMessage.addListener(event => { if (event.action === 'agent_update') relaxedFixtureUpdates.push(event); });
+ return chrome.storage.local.set({ onboardingComplete: true, pinCoachmarkPending: false, helpImproveWebBrain: false,
+ tracingEnabled: false, autoScreenshot: false, screenshotFallback: false,
+ planBeforeActMode: 'off', planBeforeAct: false, planReviewMode: 'never', maxAgentSteps: 12,
+ systemOneEnabled: false, agentAllowLocalNetwork: true, askBeforeConsequentialActions: false, openaiAskStreamingEnabled: false });
+ });
+ assert.equal((await message('update_provider', { providerId: 'openrouter', config: { apiKey: 'owned-fixture-fake-key',
+ baseUrl: `http://127.0.0.1:${server.address().port}/v1`, model: 'minimax/minimax-m3', enabled: true,
+ contextWindow: 1000000, supportsAskStreaming: false } })).ok, true);
+ assert.equal((await message('set_active_provider', { providerId: 'openrouter' })).ok, true);
+ panel = await context.newPage();
+ const panelErrors = [];
+ panel.on('pageerror', error => panelErrors.push(error.message));
+ await panel.goto(`${extensionOrigin}/src/ui/sidepanel.html`);
+ await panel.waitForFunction(() => typeof relaxedFixturePanelState === 'function');
+ await panel.evaluate(() => {
+ window.relaxedFixturePanelRequests = [];
+ const send = chrome.runtime.sendMessage.bind(chrome.runtime);
+ chrome.runtime.sendMessage = (...args) => {
+ const value = args[0];
+ if (['chat_start', 'failure_recovery_response', 'continue_start'].includes(value?.action)) relaxedFixturePanelRequests.push(structuredClone(value));
+ return send(...args);
+ };
+ });
+ for (const specification of cases) {
+ const page = await context.newPage(), pageErrors = [];
+ page.on('pageerror', error => pageErrors.push(error.message));
+ await page.goto(specification.url); await page.waitForSelector('#editor');
+ const tabId = await harness.evaluate(async url => (await chrome.tabs.query({})).findLast(tab => tab.url === url)?.id, specification.url);
+ assert.ok(tabId);
+ await page.bringToFront();
+ await panel.waitForFunction(tabId => {
+ const state = relaxedFixturePanelState(); return state.currentTabId === tabId && state.renderedTabId === tabId && !state.isProcessing;
+ }, tabId);
+ const tree = await harness.evaluate(tabId => chrome.tabs.sendMessage(tabId, { target: 'content', action: 'get_accessibility_tree', params: { filter: 'all' } }), tabId);
+ const lines = tree.pageContent.split('\n'), ref = name => lines.find(line => line.includes(`"${name}"`))?.match(/\[(ref_\d+)\]/)?.[1];
+ const refs = { editor: ref(specification.editorName), options: ref(specification.menuName) };
+ assert.ok(refs.editor && refs.options, tree.pageContent);
+ current = { ...specification, tabId, page, refs, marker: `Owned relaxed Retry scenario ${specification.name}.`,
+ phase: 'strict', requests: [], calls: [], strictDecisions: 0, retryDecisions: 0, compilers: 0, judges: 0 };
+ const probeStart = await worker.evaluate(() => ({ commands: relaxedFixtureProbe.commands.length,
+ transports: relaxedFixtureProbe.transports.length, monitorStateRequests: relaxedFixtureProbe.monitorStateRequests.length }));
+ const requestStart = await panel.evaluate(() => relaxedFixturePanelRequests.length);
+ await panel.locator('#btn-mode-act').click();
+ const task = `In this isolated owned local QA fixture, write exactly this draft into ${specification.editorName}:\n${body}\n`
+ + `Then freshly observe ${specification.menuName}, click it exactly once, and verify both the draft and the local menu. Leave all submission controls untouched. ${current.marker}`;
+ await panel.locator('#user-input').fill(task); await panel.locator('#btn-send').click();
+ const card = panel.locator('.failure-recovery-card[data-retry-mode="action_monitoring_relaxed"]');
+ await card.waitFor({ timeout: 60000 });
+ const retryButton = card.locator('[data-recovery-choice="retry"]');
+ await retryButton.waitFor({ state: 'visible' });
+ await panel.waitForFunction(() => {
+ const card = document.querySelector('.failure-recovery-card[data-retry-mode="action_monitoring_relaxed"]');
+ return card?.dataset.recoveryCurrent === 'true' && !card.querySelector('[data-recovery-choice="retry"]').disabled;
+ });
+ const strictState = await page.evaluate(() => qaSnapshot());
+ const stopped = await message('agent_run_state', { tabId });
+ const recovery = stopped.pendingFailureRecovery;
+ const strictProbe = await worker.evaluate(start => ({ commands: relaxedFixtureProbe.commands.slice(start.commands),
+ transports: relaxedFixtureProbe.transports.slice(start.transports) }), probeStart);
+ assert.equal(current.providerError, undefined, current.providerError);
+ assert.equal(stopped.running, false); assert.equal(current.strictDecisions, 5);
+ assert.equal(stopped.runUi?.status, 'failed'); assert.equal(stopped.runUi?.runSucceeded, false);
+ assert.equal(recovery?.code, 'action_context_unverified'); assert.equal(recovery?.retryMode, 'action_monitoring_relaxed');
+ assert.match(recovery.recoveryId, /^recovery_[a-z0-9]+$/); assert.equal(await card.count(), 1);
+ assert.equal(await panel.locator('.error-retry-btn').count(), 0, 'The explicit monitoring recovery must own the only Retry card.');
+ assert.match(await card.locator('.failure-recovery-note').innerText(), /relax.*monitoring.*once/i);
+ const strictRejected = strictProbe.transports.filter(item => item.action === 'page_monitor_validate_model' && item.result?.uncertified === true && item.result.ready === false);
+ assert.equal(new Set(strictRejected.map(item => item.params.snapshotToken)).size, 5, JSON.stringify(strictProbe.transports));
+ if (specification.preparationOnly) assert.ok(strictProbe.transports.some(item => item.action === 'page_monitor_dispatch'
+ && item.result?.ready === false), 'The preparation case must hit the real last dispatch fence.');
+ else assert.equal(strictRejected.length, 5, JSON.stringify(strictProbe.transports));
+ assert.equal(strictProbe.commands.filter(command => command.method === 'Input.insertText').length, 0);
+ assert.equal(strictState.draft, ''); assert.equal(strictState.optionsClicks, 0); assert.deepEqual(strictState.events, []);
+ assert.ok(strictState.ticks > 5); assert.equal(strictState.editorSame, true); assert.equal(strictState.optionsSame, true);
+ current.phase = 'retry';
+ await retryButton.click();
+ // A second physical choice is ignored once the real handler locks the
+ // consumed card. Replayed delivery uses the original real UI requestId.
+ await retryButton.evaluate(button => button.click());
+ await panel.waitForFunction(start => relaxedFixturePanelRequests.slice(start)
+ .some(request => request.action === 'failure_recovery_response' && request.choice === 'retry'), requestStart);
+ const retryRequest = await panel.evaluate(start => relaxedFixturePanelRequests.slice(start)
+ .find(request => request.action === 'failure_recovery_response' && request.choice === 'retry'), requestStart);
+ assert.ok(retryRequest.requestId); assert.equal(retryRequest.recoveryId, recovery.recoveryId);
+ const replay = await panel.evaluate(request => chrome.runtime.sendMessage(request), retryRequest);
+ assert.equal(replay.accepted, true); assert.equal(replay.requestId, retryRequest.requestId);
+ const terminalDeadline = Date.now() + 60000;
+ let finished;
+ do {
+ finished = await message('agent_run_state', { tabId, requestId: retryRequest.requestId });
+ if (!finished.running && !finished.starting && finished.runUi?.requestId === retryRequest.requestId
+ && ['completed', 'failed', 'stopped', 'cancelled'].includes(finished.runUi.status) && finished.runUi.endedAt != null) break;
+ await new Promise(resolve => setTimeout(resolve, 90));
+ } while (Date.now() < terminalDeadline);
+ current.finished = finished;
+ assert.equal(finished.running, false, 'The retry must finish within the fixture deadline.');
+ assert.equal(finished.starting, false);
+ const completedReplay = await panel.evaluate(request => chrome.runtime.sendMessage(request), retryRequest);
+ assert.equal(completedReplay.accepted, true); assert.equal(completedReplay.requestId, retryRequest.requestId);
+ const afterReplay = await message('agent_run_state', { tabId, requestId: retryRequest.requestId });
+ assert.equal(afterReplay.running, false); assert.equal(afterReplay.starting, false);
+ assert.equal(afterReplay.runUi?.runId, finished.runUi?.runId, 'A replay after completion must keep the original retry run.');
+ const lifecycle = await worker.evaluate(tabId => ({ feedbackRunPresent: relaxedFixtureAgent._pageFeedbackRuns.has(tabId),
+ reservedRetryPresent: relaxedFixtureAgent._failureRecovery.monitoringRetries.has(tabId) }), tabId);
+ const state = await page.evaluate(() => qaSnapshot());
+ const runEvents = await harness.evaluate(tabId => relaxedFixtureUpdates.filter(event => event.tabId === tabId), tabId);
+ const probe = await worker.evaluate(start => ({ commands: relaxedFixtureProbe.commands.slice(start.commands),
+ transports: relaxedFixtureProbe.transports.slice(start.transports), monitorStateRequests: relaxedFixtureProbe.monitorStateRequests.slice(start.monitorStateRequests),
+ blocked: relaxedFixtureProbe.blocked }), probeStart);
+ const panelRequests = await panel.evaluate(start => relaxedFixturePanelRequests.slice(start), requestStart);
+ const tools = runEvents.filter(event => event.requestId === retryRequest.requestId && event.type === 'tool_result').map(event => event.data);
+ const evidence = { providerLabel, specification, task, initialTree: tree, strictState, stopped, recovery, strictRejected,
+ retryRequest, replay, completedReplay, finished, afterReplay, lifecycle, state, runEvents, tools, probe, panelRequests, panelErrors, pageErrors,
+ provider: { ...current, page: undefined }, blockedRemote };
+ await fs.writeFile(path.join(out, `${specification.name}-evidence.json`), JSON.stringify(evidence, evidenceReplacer, 2));
+ assert.equal(current.providerError, undefined, current.providerError); assert.deepEqual(pageErrors, []); assert.deepEqual(panelErrors, []);
+ assert.equal(finished.runUi?.status, 'completed', JSON.stringify(finished.runUi));
+ assert.equal(finished.runUi.kind, 'continue'); assert.notEqual(finished.runUi.runId, stopped.runUi.runId);
+ assert.equal(finished.pendingFailureRecovery, null);
+ assert.deepEqual(lifecycle, { feedbackRunPresent: false, reservedRetryPresent: false }, 'The completed retry must release its run-local grant.');
+ const complete = runEvents.filter(event => event.type === 'run_complete');
+ assert.deepEqual([...new Set(complete.map(event => event.requestId))], [stopped.runUi.requestId, retryRequest.requestId]);
+ assert.equal(panelRequests.filter(request => request.action === 'chat_start').length, 1);
+ assert.deepEqual([...new Set(panelRequests.filter(request => request.action === 'failure_recovery_response').map(request => request.requestId))], [retryRequest.requestId]);
+ assert.equal(panelRequests.filter(request => request.action === 'continue_start').length, 0);
+ assert.equal(state.draft, body); assert.equal(state.editorSame, true); assert.equal(state.optionsSame, true);
+ assert.equal(state.optionsClicks, 1); assert.equal(state.menuOpen, true); assert.ok(state.ticks > strictState.ticks);
+ const insertions = probe.commands.filter(command => command.method === 'Input.insertText');
+ assert.equal(insertions.length, specification.managed ? 1 : 0);
+ if (specification.managed) assert.equal(insertions[0].params.text, body);
+ assert.ok(state.events.some(event => event.type === 'input' && event.trusted));
+ if (specification.managed) assert.ok(state.events.every(event => event.trusted));
+ assert.equal(state.inputCount, 1, 'The owned editor receives one actual input effect.');
+ assert.equal(tools.filter(tool => tool.name === 'set_field').length, 1); assert.equal(tools.find(tool => tool.name === 'set_field')?.result.success, true);
+ assert.equal(tools.filter(tool => tool.name === 'click_ax').length, 1);
+ // Durable UI projection intentionally retains only public terminal fields.
+ assert.equal(tools.findLast(tool => tool.name === 'done')?.result.outcome, 'success');
+ assert.equal(finished.runUi.successfulDone, true); assert.equal(finished.runUi.runSucceeded, true);
+ assert.ok(current.judges > 0, 'Normal production completion evaluation must run.');
+ assert.ok(probe.monitorStateRequests.some(request => request.tabId === tabId && request.documentToken), 'Content must establish run state through get_page_monitor_state.');
+ const privateTokens = [...new Set(probe.transports.map(item => item.params?.relaxedMonitorGrantToken).filter(Boolean))];
+ assert.equal(privateTokens.length, 1, 'One retry run has one private relaxed monitoring grant.');
+ assert.notEqual(privateTokens[0], recovery.recoveryId);
+ const granted = probe.transports.filter(item => item.params?.relaxedMonitorGrantToken === privateTokens[0]);
+ assert.ok(granted.some(item => item.action === 'page_monitor_validate_model' && item.result?.ready === true));
+ assert.ok(granted.some(item => item.action === 'page_monitor_prepare' && item.params.expectedModelSnapshot), 'Native preparation keeps its model snapshot plus private grant.');
+ const publicPayloads = JSON.stringify({ provider: current.requests, auxiliaryRequests, runEvents, recovery,
+ card: await card.evaluate(element => element.outerHTML), panelRequests });
+ for (const token of privateTokens) assert.equal(publicPayloads.includes(token), false, 'The monitor grant must remain absent from provider and chat payloads.');
+ summaries.push({ scenario: specification.name, strictRejections: 5, strictValidationResponses: strictRejected.length, retryRuns: 1,
+ nativeInsertions: insertions.length, optionsClicks: state.optionsClicks, passiveTicks: state.ticks, grantPrivate: true, outcome: finished.runUi.status });
+ await page.close();
+ }
+ assert.ok(blockedRemote.every(request => ['GET', 'HEAD', 'OPTIONS'].includes(request.method)), 'No external mutation may be attempted.');
+ const blockedBackground = await worker.evaluate(() => relaxedFixtureProbe.blocked);
+ assert.ok(blockedBackground.every(request => ['GET', 'HEAD', 'OPTIONS'].includes(request.method)));
+ await fs.writeFile(path.join(out, 'summary.json'), JSON.stringify({ passed: true, installedMV3: true, realRecoveryButton: true,
+ providerLabel, summaries, externalMutations: 0, blockedRemote, blockedBackground }, null, 2));
+ console.log(JSON.stringify({ passed: true, installedMV3: true, realRecoveryButton: true, providerLabel, cases: summaries, externalMutations: 0 }));
+} catch (error) {
+ const diagnostic = { providerLabel, error: error.stack, auxiliaryRequests, provider: current ? { ...current, page: undefined } : null,
+ state: await current?.page?.evaluate(() => qaSnapshot()).catch(() => null), blockedRemote,
+ runState: current && await harness?.evaluate(tabId => chrome.runtime.sendMessage({ target: 'background', action: 'agent_run_state', tabId }), current.tabId).catch(() => null),
+ updates: await harness?.evaluate(() => relaxedFixtureUpdates).catch(() => null),
+ panelState: await panel?.evaluate(() => ({ state: relaxedFixturePanelState(), requests: relaxedFixturePanelRequests,
+ messages: document.querySelector('#messages')?.innerText })).catch(() => null),
+ probe: await worker?.evaluate(() => relaxedFixtureProbe).catch(() => null) };
+ await fs.writeFile(path.join(out, 'failure-evidence.json'), JSON.stringify(diagnostic, evidenceReplacer, 2));
+ throw error;
+} finally {
+ await context?.close(); await new Promise(resolve => server.close(resolve));
+ const resolved = path.resolve(profile);
+ assert.ok(resolved.startsWith(path.resolve(os.tmpdir()) + path.sep + 'webbrain-relaxed-monitoring-retry-mv3-'));
+ await fs.rm(resolved, { recursive: true, force: true });
+}
diff --git a/test/run-ui-scheduled-lifecycle.mjs b/test/run-ui-scheduled-lifecycle.mjs
index 4de6db3a9..beb8de130 100644
--- a/test/run-ui-scheduled-lifecycle.mjs
+++ b/test/run-ui-scheduled-lifecycle.mjs
@@ -608,6 +608,19 @@ for (const item of builds) {
assert.equal(h.context.terminalRunUiStatus('Owned task completed.', revised), 'completed');
});
+ test(`${item.build}: context and page-instability recovery are failed terminal runs rather than completed tasks`, () => {
+ const h = backgroundHarness(item);
+ for (const code of ['action_context_unverified', 'page_unstable']) {
+ const failure = { type: 'failure_recovery', data: { stage: 'action_preparation', code,
+ retryMode: code === 'action_context_unverified' ? 'action_monitoring_relaxed' : 'resume' } };
+ assert.equal(h.context.terminalRunUiStatus('The task is incomplete. Choose Retry.', [failure]), 'failed');
+ assert.equal(h.context.runUpdatesSucceeded([failure]), false);
+ assert.equal(h.context.askCompletionSucceededForBadge('The task is incomplete.', [failure]), false);
+ const revised = [failure, { type: 'steering_applied', data: { id: 'new-task' } }, { type: 'tool_result', data: done }];
+ assert.equal(h.context.terminalRunUiStatus('Completed the corrected task.', revised), 'completed');
+ }
+ });
+
test(`${item.build}: actual sidebar corrected Ask succeeds only without a current error, limit or Stop`, () => {
const panel = panelCompletionHarness(item);
const error = { type: 'error', data: { message: 'Previous task failed.' } };