Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Merge remote-tracking branch 'origin/main' into test-pr-3167-merge

Codex release-state test65be8f7ecd7d
diff --cc CHANGELOG.md
index d00dc9f09,aaa0abc41..5eae878a7
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -4,170 -4,87 +4,251 @@@ All notable changes to WebBrain are doc
This changelog was generated from the repository Git history and release tags. Versions without a Git tag are inferred from version-bump commits and the current `package.json` / browser manifest versions.
+## [40.2.0] - Unreleased
+
+### Summary
+
+- General DOM monitoring repairs prevent decorative icons, passive updates and acknowledged agent preparation from incorrectly invalidating unchanged controls, while retaining checks for real target or user changes.
+- Mid-run task changes, Stop, durable chat recovery, Telegram/WhatsApp observation, recipient ownership and completion receipts are covered by causal and installed-browser regressions.
+- File download/upload ownership, literal editor input, exact calculation and complete literal table delivery are repaired across Chrome and Firefox.
+- The original audit passed 9,765 Node/main cases and all 21 installed browser stages; coverage was 65.16% lines/statements, 78.74% branches and 87.25% functions. The Codex review follow-up additionally passed 72 focused, 155 neighbouring and 2,450 main regression cases. Both browser builds passed.
+- Real-account two-hour endurance remains unverified. The first fresh PR CodeQL findings check flagged 32 changed-code alerts (31 high, one medium); the comparison and directly related QA fixture repair are recorded in the appendix. This entry records the proposed release; the audited runtime remains 40.0.8.
+
+The complete dated audit, including earlier failures, tracked effects and verification limits, is copied in the [40.2.0 appendix at the end of this changelog](#appendix-4020-quality-audit). A standalone copy is available in [the audit report](docs/quality-audit-2026-10-07.md).
+
+### Added
+- Added bounded exact decimal arithmetic through a read-only `calculate` tool.
+- Added native Telegram observation and delivery verification, with isolated installed-extension regression tests for generated conversations.
+- Added native WhatsApp one-to-one contact observation and opaque-ID delivery proof from a freshly verified contact drawer and retained pre-send transcript nodes.
+- Added a full-suite Node coverage command and optional live-model browser checks.
+
+### Fixed
+- Share concurrent cold UI snapshot reads so service-worker restart and continuation cannot discard saved events or progress; preserve clear/new-run invalidation.
+- Reject injected URL parameters in the disposable workflow QA server without changing valid fixtures or weakening replay checks.
+- Keep Enter-dispatch uncertainty truthful when reusing an owned chat draft, without weakening pending-send or duplicate-send guards.
+- Preserve exact native commit acknowledgement provenance for independent completion reconciliation after interrupted observation.
+- Bind planner reference-history repair to the current task, permit at most one verified replan, and recheck Stop/steering authority.
+- Execute one-use native chat read preparation through guarded tools, retire stale preparation when tasks change, and report unscheduled partial endings truthfully in all 23 UI languages.
+- Preserve complete table JSON, literal Unicode and row order through fresh source verification, terminal delivery, persistence and cold UI replay; reject oversized complete output before issuing a reference.
+- Correct privacy/default-sharing disclosures across 22 localized landing pages, and retain the website JSON script-boundary hardening tests.
+- Matched spreadsheet services and social image upgrades against parsed URL hosts and supported paths, preventing unrelated path or query text from selecting a service or changing an image URL.
+- Replaced ambiguous grouped-recipient and bulletin-comment matching with bounded linear parsing, preserving recipient role, negation and content checks.
+- Made the bundled BitGPU XML tool parser consume complete blocks in order and treat declared arguments as own data properties; generation template preprocessing now avoids repeated whitespace backtracking.
+- Bound hidden file uploads to a fully observed native section and its distinct visible purpose, distinguishing repeated Upload controls without site-specific rules.
+- Preserved a verified native chat draft when explicit submission is blocked before Enter; a fresh observation and new model-approved send can submit the same owned draft once, while actual or ambiguous submissions remain pending.
+- Bound complete native navigation, banner and tab-list contexts across all sites, preserving unchanged actions during unrelated passive updates while retaining account, target and user-intervention checks.
+- Preserved the bound action when a noninteractive decorative SVG hit is replaced, while rejecting changed controls, handlers, recipients and overlays.
+- Distinguished an initially unverified action context from an actual page change, retaining bounded fresh-observation recovery without dispatching an uncertified action.
+- Kept hidden noninteractive icon labels out of model-visible dialog identity while preserving target semantics, handler checks and harmless navigation badge updates.
+- Recovered native Telegram username routes through exact numeric peer evidence and narrowly guarded read-only menu, Settings and profile preparation.
+- Initialized cold native read-only preparation only inside the current private capture and preserved its one-use proof through actual guarded dispatch.
+- Required a verified latest Telegram viewport and one shared highest-200 mounted native ID floor, including media-only rows; refused changed observation-to-probe baselines before persisting or dispatching a structured send.
+- Restored supported native chat tools in the full planner catalog and kept literal calculation independent of passive page updates.
+- Accepted completion of a current, exclusive arithmetic task from its actual owned calculation result, while preserving mixed-task, pending-action and Stop checks.
+- Matched arithmetic expressions through the calculator's shared lexer and bounded parser, allowing whitespace between exact tokens while rejecting split numbers, changed inputs and mixed tasks.
+- Retained plain responses for a current arithmetic-only task during passive page updates so normal recovery can demand the actual `calculate` call and explicit `done`; plain text never creates calculation evidence or success.
+- Kept exclusive arithmetic tasks on `calculate` and `done`, projecting `done` alone after the current calculation receipt so unrelated browser actions cannot derail completion.
+- Verified explicit arithmetic completion against the actual owned result and recognized output requirements; unsupported formats retain the ordinary evaluator with bounded calculation evidence.
+- Rechecked original task, provider, receipt, run and document ownership across completion awaits and errors, and kept incorrect result or marker summaries repairable without creating browser verification debt.
+- Allowed positively resolved ordinary social UI controls for current nonpublication tasks without weakening Send/Publish contracts.
+- Verified native Chrome contenteditable clear-and-replace operations through an exact temporary deletion receipt while preserving authored line breaks and user-intervention checks.
+- Restored native WhatsApp contact verification for generic, unfocused composers and supplied exact contact-name and Close references for fresh accessibility reads and guarded identity repair.
+- Preserved native WhatsApp identity through unrelated contact-drawer media and Close-icon updates, while requiring unchanged original identity nodes and a guarded native Close receipt.
+- Explained historical WhatsApp message views with an exact message-pane reference for scoped scrolling and fresh observation before sending.
+- Prioritized exact private WhatsApp repair targets in the normal accessibility observation before inference, avoiding an extra model read for a current native contact-name control, captured Close button or historical message pane.
+- Kept manual and queued messages bound to their original tab and draft across asynchronous work.
+- Preserved accepted mid-run corrections through terminal recovery, and kept Stop ahead of pending corrections.
+- Restored current unanswered permission and clarification cards after acknowledged progress or panel reopening, with live request, task and provider ownership checks.
+- Woke obsolete permission waits when a mid-run correction is accepted, without granting the earlier permission or dispatching its action.
+- Made startup cancellation, browser action deadlines, and persisted tool batches reliable.
+- Verified the submitted form and fresh acknowledgements without letting unrelated live counters invalidate successful completion.
+- Preserved completed bulk-work progress across Continue and retried only unfinished items.
+- Preserved literal multiline rich text, and fenced iframe preparation and dispatch against Stop and intervening user edits.
+- Replaced session-changing PDF HEAD probes with document-owned MIME evidence.
+- Kept shortcut key releases inside the shortcut even when submitting closes it before keyup.
+- Persisted native chat delivery and message watermarks to prevent duplicate replies and stale delivery replay after worker restarts.
+- Kept nested chat sends, ordinary guarded sends and terminal completion evidence consistent.
+- Removed false chat loop warnings, bounded repeated unchanged page retries, and made recovery text processing linear.
+- Kept trusted task anchors separate from injected page context and guarded malformed cloud bridge messages.
+- Bound MemCode connections, refreshes and late recall results to the current account and opt-in state.
+- Fixed strict planner schema compatibility and kept protocol repairs separate from the genuine user task.
+- Kept chained chat-resume alarms distinct from their running predecessor, and restored fresh native delivery proof after worker restart.
+- Counted generic chat submissions from exact fresh native delivery receipts and reconciled only the matching single-conversation reply ledger.
+- Recognized Mastodon's rendered decorative privacy icons while retaining hidden-control and document ownership checks.
+- Shared provider message projection and kept private resume metadata out of external request payloads.
+- Re-read the exact accessibility subtree across model decisions so offscreen controls remain safely targetable.
+- Preserved native chat empty-editor receipts without discarding authored line breaks, and kept nested blocked-send recovery instructions intact.
+- Made same-peer observation echoes idempotent without resetting pending delivery state.
+- Exposed rendered native dropdown actions below the viewport while preserving dispatch ownership checks.
+- Retained exact native commit ownership through an explicit same-origin verification navigation and a fresh independent verdict.
+- Kept delayed chat delivery receipts attached to their original question when a new question needs the same reply body.
+- Preserved the latest unread tool result through context compaction, including its complete tool-call group and task authority.
+- Checked ancestor visibility for native confirmation actions, and excluded invisible duplicate modals and primary buttons from their ownership checks.
+- Left bounded output room for thinking models to produce the independent completion verdict.
+- Captured verification navigation at native document commit so delayed content-script readiness cannot discard the original action proof.
+- Included the actual tool schemas in first-request context sizing and corrected the onboarding context-window guidance.
+- Recognized the configured active provider during onboarding, kept stale local scans from replacing it, and made “Skip for now” actionable when no local model is found.
+- Awaited local onboarding persistence before focusing the composer.
+- Recovered saved workflow targets from native AX attributes without selecting a reused ID that contradicts the recorded field name.
+- Settled watch checkpoints from matching fresh independent completion evidence and rechecked Stop and accepted corrections after terminal persistence.
+- Preserved literal editor line breaks and rejected incomplete native DraftJS reads.
+- Bound complete read-only dialog labels while rejecting nested clickable or focusable identity displays.
+- Removed stale “running” wording from the static plan-approval note.
+- Required a one-use user answer before reconciling an uncertain outgoing message, with exact pending-record, task, provider and conversation ownership checks.
+- Recovered an extension-owned Chrome debugger after a service-worker restart, while rejecting foreign ownership, permission failures and late replies after detach or Stop.
+- Preserved one-to-one WhatsApp send evidence only across the same genuine task, provider, account and document; Stop, a new task and lost transcript evidence retain an uncertain send without resending.
+- Verified exact native page-handled drafts independently of unrelated DOM updates, while preserving stale-action refusal and pinning the original chat identity nodes.
+- Persisted exact scheduled completion outcomes before snapshot cloning, and kept late terminal or restored UI events from overwriting a resumed or newer task.
+- Classified completion and notifications against the latest accepted task correction, preserving earlier events in history while preventing an old success or error from replacing the corrected task's result.
+
+## [40.0.8] - 2026-10-07
+
+### Added
+- Added upstream regression coverage for trusted field preparation and dialog control labels.
+
+## [40.0.7] - 2026-10-07
+
+### Fixed
+- Tightened passive page feedback for dialogs, external control labels and tool-free responses.
+- Distinguished unchanged trusted editor preparation from an uncertain dispatched text write.
+
+## [40.0.6] - 2026-10-07
+
+### Fixed
+- Preserved recipient approval across Continue and reported missing draft or delivery evidence as a technical blocker instead of repeatedly asking for the same approval.
+- Updated WhatsApp composer, outgoing message, and recipient-bound dispatch checks for the current web client.
+
+## [40.0.5] - 2026-10-07
+
+### Fixed
+- Kept browser actions progressing when unrelated page updates arrive during observation and dispatch.
+- Tracked finite collection reads so previously unseen ranges advance the task without allowing repeated reads to reset progress.
+- Preserved photo download bindings across harmless page updates and prevented browser feedback from starving task progress.
+- Retried task completion safely when passive page updates arrive.
+
+## [40.0.2] - 2026-10-07
+
+### Fixed
+- Improved recovery and progress tracking after the page feedback monitoring rollout.
+
+## [40.0.1] - 2026-10-07
+
+### Changed
+- Updated the release artwork and refreshed the 40.0.0 release materials.
+
+## [40.0.0] - 2026-10-07
+
+### Added
+- Added page feedback monitoring across forms, navigation, frames, accessibility state, and browser automation, with consent based trace snapshots for GitHub feedback drafts.
+- Added decision model verification for task completion using fresh, run scoped page evidence and safer handling of cost limits in scheduled jobs.
+- Added Ling 3.1 Flash planner benchmarks and a historical planner comparison panel for Ling and Nex.
+
+### Changed
+- Improved recovery from rate limited browser inference without replaying completed tool actions.
+- Improved Firefox BiDi and CDP action attribution, stale page feedback fencing, and WebMCP registration recovery.
+- Added a Turkish README translation and refreshed planner benchmark materials.
+
+### Fixed
+- Fixed page feedback attribution and monitoring for keyboard actions, shadow DOM, forms, redirects, stylesheets, frames, and animated page changes.
+- Fixed completion evidence freshness and scope so unrelated or stale page changes cannot satisfy a run.
+
- ## [39.2.0] - 2026-10-07
+ ## [39.2.2] - 2026-10-10
+
+ ### Changed
+ - test(feedback): wait safely for asynchronous submission result
+ - fix(captcha): resolve Firefox bridge script from extension root
+ - fix(captcha): persist bridge registration across restarts
+ - fix(captcha): scope callback bridge to enabled providers
+
+ ## [39.2.1] - 2026-10-10
+
+ ### Changed
+ - chore: bump version to 39.2.0
+ - fix(captcha): preserve callback completion and retries
+ - feat(providers): add Claude Sonnet 5.5 and Haiku 5.5 support to Anthropic provider
+ - docs(faq): localize subscription-unrecognized, emergency-box and offline-licensing in all locales
+ - docs(faq): add paid-but-asks-again subscription entry, promote workflow-editor to locale sources
+
+ ## [39.2.0] - 2026-10-10
+
+ ### Fixed
+ - Preserve CAPTCHA completion callbacks for widgets rendered during page load in Chrome and Firefox.
+ - Prevent callback retries from replaying response-field events or replacing a successful field application.
+ - Install the Firefox callback bridge in the detected inherited-origin descendant frame.
+
+ ## [39.1.4] - 2026-10-09
+
+ ### Changed
+ - Complete release locale coverage and requested local provider label
+ - Remove retired CAPTCHA locale keys required by release checks
+ - Match local WebGPU bundle to its native runtime factory
+ - Pin renamed d1 browser decision package
+ - Use service-worker-safe imports and guarded local decision diagnostics
+ - Add opt-in pinned FP32 local decision provider
+ - feat(providers): support custom titles and unlimited duplicates
+ - Exclude mutable accounting metadata from feedback snapshots
+ - Freeze feedback traces when the rating prompt opens
+ - Publish Ling 3.0 Flash VL budget vision benchmark and raw results
+ - Attach feedback traces to GitHub drafts with explicit consent
+ - Bound rejected WebMCP registration retries and explain DOM recovery
+ - Add Ling and Nex historical planner panel and prepare matched vision replay
+ - fix(agent): propagate completion cost stops to scheduled jobs
+ - Add Ling 3.1 Flash planner benchmark and blog comparison
+ - fix(agent): isolate completion evidence to the active run
+ - fix(agent): scope completion freshness to shared evidence
+ - fix(agent): fingerprint complete UTF-16 document text
+ - fix(agent): verify moving visual evidence with fresh AX
+ - fix(agent): preserve completion fallback and evidence freshness
+ - feat(agent): verify completion with decision models and fresh evidence
+ - Recover rate-limited browser inference without replaying tools
+ - Consume final-step steering before terminating a run
+ - Reset superseded skills before steering reauthorization
+ - Hide unconfigured media generation tools and guidance
+ - Skip superseded planner recommendations and allow form-free image previews
+ - Preserve recovery task hashes and recognize localized LinkedIn previews
+ - Preserve steering through emergency trims and initial responses
+ - version up
+ - build(deps): bump @modelcontextprotocol/sdk in /mcp-server
+ - Fix media service-worker imports and support OpenRouter videos
+ - Fix steering reauthorization and LinkedIn message inspection
+ - version up
+ - Add OpenRouter and Comfy media providers
+ - Add Turkish README translation for WebBrain
+ - version up
+ - fix: accept successful fal.ai generation as completion evidence
+ - fix: honor GPT-6.1 and Opus 5.5 contracts
+ - feat: add GPT-6.1 Sol and Claude Opus 5.5
+ - Unify bridge controls in Bridge tab with preset pills, queue shortcut, and full localization
+ - Translate composer delivery setting in all locales
+ - Simplify steering controls and add default delivery setting
+ - chore: release v39.0.0
+ - build(deps): bump multidict
+ - build(deps): bump multidict
+ - build(deps): bump proxy-addr
+ - build(deps): bump proxy-addr from 2.0.7 to 2.0.8 in /mcp-server
+ - Add optional steering to active chat runs
+ - Benchmark Ling 3.0 Flash VL against Nex-N2.5 and MiniMax M3
+ - fix: harden fal.ai media generation
+ - chore: uncomment the .gitignore
+ - fix: match source-inspection regexes in chrome background.js
+ - feat: add fal.ai generative media tool to the assistant
+
++## [39.2.0-preview] - 2026-10-07
+
+### Added
+- Added generative media creation with fal.ai, OpenRouter, and Comfy providers, including video support and safer provider handling.
+- Added optional MemCode OAuth recall and controls for steering active runs, including queued follow-ups and localized Bridge settings.
+- Added GPT-6.1 Sol and Claude Opus 5.5 provider support, plus Ling and Nex planner comparison tools.
+
+### Changed
+- Improved completion verification with fresh page evidence, run-scoped results, and clearer handling of cost limits in scheduled jobs.
+- Improved steering recovery and authorization, including final-step corrections and LinkedIn message inspection.
+- Updated Turkish documentation and refreshed media and planner benchmark materials.
+
+### Fixed
+- Fixed media generation tool visibility when providers are not configured and corrected service-worker imports for media providers.
+- Improved recovery from superseded planner recommendations and recognition of localized LinkedIn previews.
+
## [39.0.0] - 2026-10-06
### Added
diff --cc package.json
index caf68dc36,b49510ec4..3ed287c5c
--- a/package.json
+++ b/package.json
@@@ -5,16 -5,16 +5,16 @@@
"private": true,
"type": "module",
"scripts": {
- "test": "npm run test:cms && npm run test:markdown && npm run test:generative-media && npm run test:systemone && npm run test:systemone:fast && node --test test/browser-dialogs.mjs test/capsolver-cloud-broker.mjs test/captcha-providers.mjs test/captcha-weighted-fallback.mjs test/captcha-hcaptcha-providers.mjs test/captcha-native-providers.mjs test/captcha-runtime-policy.mjs test/captcha-callback-bridge.mjs test/captcha-aws-recovery.mjs test/captcha-documented-contracts.mjs test/huggingface-signup-recovery.mjs test/trace-full-export.mjs test/feedback-traces.mjs && npm run test:firefox-bidi && npm run test:runtime-lifecycle && npm run test:billing-notice && npm run test:compass-quota && npm run test:provider-limits && npm run test:provider-catalog && npm run test:provider-instances:ui && npm run test:openrouter-capabilities && npm run test:anthropic && npm run test:accessibility-tree-benchmark && npm run test:toolbar-guard && npm run test:cloud-bridge-approval && npm run test:discord && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && npm run test:safesocial && npm run test:build-unpacked && npm run test:attachment-drop && npm run test:workflow-editor && node --test test/profile-sync-stats.mjs && node --test test/memcode-recall.mjs && node test/run.js && node test/selection-scope-restoration.mjs && node scripts/benchmark-offline-relevance.mjs && npm run test:security",
- "test": "npm run test:cms && npm run test:markdown && npm run test:generative-media && npm run test:systemone && npm run test:systemone:fast && node --test test/browser-dialogs.mjs test/capsolver-cloud-broker.mjs test/captcha-providers.mjs test/captcha-weighted-fallback.mjs test/captcha-hcaptcha-providers.mjs test/captcha-native-providers.mjs test/captcha-runtime-policy.mjs test/captcha-callback-bridge.mjs test/captcha-bridge-registration.mjs test/captcha-aws-recovery.mjs test/captcha-documented-contracts.mjs test/huggingface-signup-recovery.mjs test/trace-full-export.mjs test/feedback-traces.mjs && npm run test:firefox-bidi && npm run test:runtime-lifecycle && npm run test:billing-notice && npm run test:compass-quota && npm run test:provider-limits && npm run test:provider-catalog && npm run test:openrouter-capabilities && npm run test:accessibility-tree-benchmark && npm run test:toolbar-guard && npm run test:cloud-bridge-approval && npm run test:discord && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && npm run test:safesocial && npm run test:build-unpacked && npm run test:attachment-drop && npm run test:workflow-editor && node --test test/profile-sync-stats.mjs && node --test test/memcode-recall.mjs && node test/run.js && node test/selection-scope-restoration.mjs && node scripts/benchmark-offline-relevance.mjs && npm run test:security",
++ "test": "npm run test:cms && npm run test:markdown && npm run test:generative-media && npm run test:systemone && npm run test:systemone:fast && node --test test/browser-dialogs.mjs test/capsolver-cloud-broker.mjs test/captcha-providers.mjs test/captcha-weighted-fallback.mjs test/captcha-hcaptcha-providers.mjs test/captcha-native-providers.mjs test/captcha-runtime-policy.mjs test/captcha-callback-bridge.mjs test/captcha-bridge-registration.mjs test/captcha-aws-recovery.mjs test/captcha-documented-contracts.mjs test/huggingface-signup-recovery.mjs test/trace-full-export.mjs test/feedback-traces.mjs && npm run test:firefox-bidi && npm run test:runtime-lifecycle && npm run test:billing-notice && npm run test:compass-quota && npm run test:provider-limits && npm run test:provider-catalog && npm run test:provider-instances:ui && npm run test:openrouter-capabilities && npm run test:anthropic && npm run test:accessibility-tree-benchmark && npm run test:toolbar-guard && npm run test:cloud-bridge-approval && npm run test:discord && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && npm run test:safesocial && npm run test:build-unpacked && npm run test:attachment-drop && npm run test:workflow-editor && node --test test/profile-sync-stats.mjs && node --test test/memcode-recall.mjs && node test/run.js && node test/selection-scope-restoration.mjs && node scripts/benchmark-offline-relevance.mjs && npm run test:security",
"test:captcha:ui": "node test/captcha-settings-ui.mjs && node test/captcha-application-ui.mjs",
"test:captcha:bridge:ui": "node test/captcha-callback-bridge-ui.mjs",
"test:billing-notice": "node test/billing-payment-notice.mjs",
"test:provider-limits": "node test/provider-model-limits.mjs",
"test:provider-catalog": "node test/provider-catalog-integrations.mjs",
"test:provider-instances:ui": "node test/provider-instances-ui.mjs",
- "test:openrouter-capabilities": "node --test test/openrouter-model-capabilities.mjs test/muse-tool-choice.mjs test/model-retry.mjs",
+ "test:openrouter-capabilities": "node --test test/openrouter-model-capabilities.mjs test/muse-tool-choice.mjs test/qwen-tool-calls.mjs test/dolphin-prompted-tools.mjs test/model-retry.mjs",
"test:markdown": "node --test test/markdown-render.mjs",
- "test:accessibility-tree-benchmark": "node --test test/llm/lib/accessibility-tree-formats.test.mjs && node test/llm/accessibility-tree-benchmark.mjs --no-exact-tokenizer --check",
+ "test:accessibility-tree-benchmark": "node --test test/llm/lib/accessibility-tree-formats.test.mjs test/llm/report-html-encoding.mjs && node test/llm/accessibility-tree-benchmark.mjs --no-exact-tokenizer --check",
"test:attachment-drop": "node test/attachment-drop.mjs",
"test:workflow-editor": "node --test test/workflow-editor.mjs",
"test:security": "node test/security/injection-corpus.mjs",
diff --cc src/chrome/manifest.json
index 8e1401354,0bf76821f..4f3b68db5
--- a/src/chrome/manifest.json
+++ b/src/chrome/manifest.json
@@@ -46,23 -46,6 +46,23 @@@
"type": "module"
},
"content_scripts": [
+ {
+ "matches": ["<all_urls>"],
+ "js": ["src/content/page-monitor.js"],
+ "run_at": "document_start",
+ "all_frames": true,
+ "match_about_blank": true,
+ "match_origin_as_fallback": true
+ },
+ {
+ "matches": ["<all_urls>"],
- "js": ["src/content/page-monitor-shadow.js", "src/content/captcha-callback-bridge.js"],
++ "js": ["src/content/page-monitor-shadow.js"],
+ "run_at": "document_start",
+ "world": "MAIN",
+ "all_frames": true,
+ "match_about_blank": true,
+ "match_origin_as_fallback": true
+ },
{
"matches": ["<all_urls>"],
"js": ["src/content/file-picker-guard-page.js"],
diff --cc src/chrome/src/agent/agent.js
index b13dc9e6f,12e6240c9..89ab51c93
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@@ -12,12 -6,9 +12,13 @@@ import { JEV_FAST_KEYS, JEV_CLASSIFIER_
import { redactSystemOneText, wrapSystemOneData, boundedSystemOneText } from './systemone-evidence.js';
import { createSystemOneJudge, isSystemOneResponseContractError, systemOneFailureReason, SYSTEM_ONE_COST_PROVIDER } from './systemone-judge.js';
import { SOCIAL_PLATFORMS, socialPublicationApiPlatform, normalizePublicationContract, publicationProgress, exactPublicationText, publicationMediaMatches, publicationContractMessages, publicationAuditMessages, publicationAuditAccepted } from './social-publish-contract.js';
+import { mastodonPublicationIntentMessages, normalizeMastodonPublicationIntent, captureNativeMastodonPublication, mastodonPublicationBinding, mastodonPublicationReceipt, mastodonPublicationNavigation } from './mastodon-publication.js';
import { AGENT_TOOLS, AGENT_TOOL_NAMES, RESERVED_AGENT_TOOL_NAMES, getToolsForMode, SYSTEM_PROMPT_ASK, SYSTEM_PROMPT_ACT, SYSTEM_PROMPT_ACT_COMPACT, SYSTEM_PROMPT_ACT_MID, SYSTEM_PROMPT_GENERATIVE_MEDIA, SYSTEM_PROMPT_DEV_APPENDIX, SYSTEM_PROMPT_WEBMCP_ASK, SYSTEM_PROMPT_WEBMCP_ACT } from './tools.js';
+ import { rejectedCompletionRecovery } from './completion-recovery.js';
import { validateToolArguments } from './tool-arguments.js';
+import { calculateExpression, calculationResultInText } from './calculator.js';
+import { isLiteralCalculationTask, literalCalculationSummaryFormatMatches, isPassivePageFeedback } from './page-feedback-policy.js';
+import { chatRecoveryDisplayText, terminalRecoveryFinalText } from './chat-recovery-text.js';
import { isSessionQuotaError, serializeConversationForSession, SESSION_CONVERSATION_BUDGET_BYTES, SESSION_CONVERSATION_RETRY_BUDGET_BYTES } from './conversation-persistence.js';
import { formatErrorMessage } from '../error-format.js';
import { retryModelCall } from '../providers/model-retry.js';
@@@ -9633,29 -8872,49 +9641,70 @@@ export class Agent extends LoopDetecto
}
_repairToolCallArgs(name, args = {}, options = {}) {
+ if (name === 'read_page' && args && typeof args === 'object' && !Array.isArray(args)) {
- const next = { ...args };
++ let next = args;
+ let repaired = false;
- const boundWindow = (window) => {
- if (Number.isInteger(window.limit) && window.limit > 6000) {
- window.limit = 6000;
++ if (Number.isInteger(args.limit) && args.limit > 6000) {
++ next = { ...args, limit: 6000 };
++ repaired = true;
++ }
++ if (args.continuationArgs && typeof args.continuationArgs === 'object' && !Array.isArray(args.continuationArgs)) {
++ if (Number.isInteger(args.continuationArgs.limit) && args.continuationArgs.limit > 6000) {
++ next = { ...(repaired ? next : args), continuationArgs: { ...args.continuationArgs, limit: 6000 } };
+ repaired = true;
+ }
- };
- boundWindow(next);
- if (next.continuationArgs && typeof next.continuationArgs === 'object' && !Array.isArray(next.continuationArgs)) {
- next.continuationArgs = { ...next.continuationArgs };
- boundWindow(next.continuationArgs);
+ }
- return { args: next, repaired, note: repaired
- ? '[TOOL ARGUMENT REPAIR: Bounded read_page limit to its advertised maximum of 6000 characters. The requested offset and extraction options are preserved; continue with the exact returned continuationArgs when hasMore is true.]'
- : '' };
++ return {
++ args: repaired ? next : args,
++ repaired,
++ note: repaired
++ ? '[TOOL ARGUMENT REPAIR: Bounded read_page limit to its advertised maximum of 6000 characters. The requested offset and extraction options are preserved; continue with the exact returned continuationArgs when hasMore is true.]'
++ : '',
++ };
+ }
- if (name !== 'get_accessibility_tree' || !args || typeof args !== 'object' || Array.isArray(args)) {
+ if (!['get_accessibility_tree', 'wait_for_stable'].includes(name) || !args || typeof args !== 'object' || Array.isArray(args)) {
return { args, repaired: false, note: '' };
}
- const next = { ...args };
+ let next = { ...args };
let repaired = false;
+ const decimal = value => typeof value === 'string' && /^-?(?:0|[1-9]\d*)(?:\.\d+)?$/.test(value.trim())
+ && Number.isFinite(Number(value.trim())) ? Number(value.trim()) : value;
+ if (name === 'wait_for_stable') {
+ for (const key of ['quietMs', 'timeout']) {
+ const value = decimal(next[key]);
+ if (value !== next[key]) { next[key] = value; repaired = true; }
+ }
+ if (next.checkNetwork === 'true' || next.checkNetwork === 'false') {
+ next.checkNetwork = next.checkNetwork === 'true';
+ repaired = true;
+ }
+ return { args: next, repaired, note: repaired ? '[TOOL ARGUMENT REPAIR: Normalized declared wait_for_stable scalar types. Schema validation still applies.]' : '' };
+ }
+ if (Object.hasOwn(next, 'continuationArgs')) {
+ let continuation = next.continuationArgs;
+ if (typeof continuation === 'string') {
+ try { continuation = continuation.length <= 32768 ? JSON.parse(continuation) : null; } catch { continuation = null; }
+ }
+ const allowed = ['filter', 'maxDepth', 'maxChars', 'ref_id', 'page', 'tree_revision'];
+ const valid = continuation && typeof continuation === 'object' && !Array.isArray(continuation)
+ && Object.keys(continuation).length > 0
+ && Object.keys(continuation).every(key => allowed.includes(key))
+ && Object.entries(continuation).every(([key, value]) => !Object.hasOwn(next, key) || Object.is(next[key], value));
+ if (!valid) {
+ return { args, repaired: false, note: '', result: {
+ success: false, invalidArguments: true, invalidToolArguments: true,
+ noDispatch: true, dispatched: false, errorCode: 'invalid_tool_arguments',
+ invalidArgumentNames: ['$.continuationArgs'],
+ error: 'get_accessibility_tree continuationArgs must be one complete JSON object containing only declared read fields, with no conflicting top-level fields. Re-emit the exact returned continuation fields as top-level arguments.',
+ } };
+ }
+ next = this._normalizeContinuationToolArgs(name, { ...next, continuationArgs: continuation });
+ repaired = true;
+ }
+ for (const key of ['maxDepth', 'maxChars', 'page']) {
+ const value = decimal(next[key]);
+ if (value !== next[key]) { next[key] = value; repaired = true; }
+ }
const filter = String(next.filter || '').trim();
if (filter && !/^(?:all|visible|interactive)$/i.test(filter)) {
const match = filter.match(/\b(all|visible|interactive)\b/i);
@@@ -46639,8 -43855,10 +46748,12 @@@ If the user has already named or confir
if (completionRecoveryPolicy) {
tools = completionRecoveryPolicy.tools;
}
+ const invalidArgumentToolChoice = !invalidArgumentRecoveryUsed && !completionRecoveryPolicy
+ ? this._invalidArgumentRecoveryToolChoice(provider, messages, tools) : null;
- const completionToolChoice = completionRecoveryPolicy?.toolChoice || invalidArgumentToolChoice;
+ const completionToolChoice = completionRecoveryPolicy?.toolChoice
- || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null) || null;
++ || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null)
++ || invalidArgumentToolChoice || null;
+ if (invalidArgumentToolChoice) invalidArgumentRecoveryUsed = true;
allowedToolNames = new Set(tools.map(t => t.function.name));
toolSchemas = new Map(tools.map(t => [t.function.name, t.function.parameters]));
@@@ -47908,8 -45082,10 +48028,12 @@@
if (completionRecoveryPolicy) {
tools = completionRecoveryPolicy.tools;
}
+ const invalidArgumentToolChoice = !invalidArgumentRecoveryUsed && !completionRecoveryPolicy
+ ? this._invalidArgumentRecoveryToolChoice(provider, messages, tools) : null;
- const completionToolChoice = completionRecoveryPolicy?.toolChoice || invalidArgumentToolChoice;
+ const completionToolChoice = completionRecoveryPolicy?.toolChoice
- || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null) || null;
++ || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null)
++ || invalidArgumentToolChoice || null;
+ if (invalidArgumentToolChoice) invalidArgumentRecoveryUsed = true;
allowedToolNames = new Set(tools.map(t => t.function.name));
toolSchemas = new Map(tools.map(t => [t.function.name, t.function.parameters]));
@@@ -47942,8 -45117,8 +48066,9 @@@
let reasoningContent = '';
let streamUsage = null;
let finishReason = '';
+ let rejectedToolResponse = null;
+ await this._capturePageFeedbackModelState(tabId, pendingVisionFallbackMessages || messages);
const fastResult = await this._maybeJevFastTurn(tabId, userMessage, messages, mode, allowedToolNames, provider, costState, runOptions, completionRecoveryPolicy);
const streamOpts = this._cloudGenerationOptions(provider, {
signal: this._runAbortSignal(tabId),
diff --cc src/chrome/src/agent/tool-arguments.js
index 788549254,a7f300726..8ae52dac9
--- a/src/chrome/src/agent/tool-arguments.js
+++ b/src/chrome/src/agent/tool-arguments.js
@@@ -186,16 -181,11 +186,19 @@@ export function validateToolArguments(t
const failures = [];
validateValue(normalizedArgs, closedParameters, '$', failures);
if (failures.length) {
- return validationFailure(toolName, failures, `Invalid or undeclared argument(s): ${[...new Set(failures)].join(', ')}.`);
+ const enumGuidance = Object.entries(closedParameters.properties || {})
+ .filter(([key, schema]) => failures.includes(`$.${key}`) && Array.isArray(schema.enum))
+ .map(([key, schema]) => `${key} must be one of ${JSON.stringify(schema.enum)}.`).join(' ');
+ return validationFailure(toolName, failures, `Invalid or undeclared argument(s): ${[...new Set(failures)].join(', ')}. ${enumGuidance}`.trim());
}
+ if (toolName === 'get_accessibility_tree') {
+ const refs = [['$.ref_id', normalizedArgs.ref_id],
+ ['$.continuationArgs.ref_id', normalizedArgs.continuationArgs?.ref_id]];
+ const invalid = refs.filter(([, value]) => typeof value === 'string'
+ && AX_REF_ID_REGEX.exec(value)?.[0] !== value).map(([path]) => path);
+ if (invalid.length) return validationFailure(toolName, invalid,
+ 'Use an exact observed native ref_id (ref_ followed by digits), or omit ref_id for a root read. Do not append JSON punctuation or invent an anchor.');
+ }
if (toolName === 'click') {
const clickFailure = validateClickTarget(normalizedArgs);
if (clickFailure) return clickFailure;
diff --cc src/chrome/src/cloud-runs.js
index dccbfd5e8,56f3cb1b6..ab43066d8
--- a/src/chrome/src/cloud-runs.js
+++ b/src/chrome/src/cloud-runs.js
@@@ -1080,8 -1070,10 +1091,10 @@@ export function createCloudRunControlle
finally { if (key != null) startingConversations.delete(key); }
}
- async function startRunReserved(msg = {}) {
+ async function startRunReserved(msg = {}, startup = null) {
await hydrate();
+ const recipient = msg.private_result_public_key == null
+ ? null : await importPrivateResultRecipient(msg.private_result_public_key);
const suppliedRunId = msg.runId ?? msg.run_id;
const requestedRunId = suppliedRunId == null ? '' : String(suppliedRunId).trim();
const parentRunId = String(msg.parentRunId || msg.parent_run_id || '').trim() || null;
@@@ -1194,22 -1198,12 +1226,24 @@@
completedAt: null,
};
runs.set(run.runId, run);
+ startingRunIds.delete(runId);
+ if (recipient) privateRecipients.set(run.runId, recipient);
+ const runController = new AbortController();
+ runAbortControllers.set(runId, runController);
+ const cancelled = () => runController.signal.aborted;
+ const checkCancellation = () => {
+ if (cancelled()) throw new Error('Abort requested.');
+ };
+ const cancellationOptions = {
+ signal: runController.signal,
+ isDetachedStartCancelled: cancelled,
+ };
try {
await persist();
} catch (error) {
runs.delete(run.runId);
+ runAbortControllers.delete(runId);
+ privateRecipients.delete(run.runId);
startingTabs.delete(tabId);
throw error;
}
@@@ -1407,10 -1397,29 +1443,29 @@@
}
}
}
+ if (recipient && run.status !== 'aborted') {
+ const raw = structured ? privateStructuredResults.get(run.runId) : content;
+ if (raw !== undefined) {
+ // Encryption is asynchronous. Pollers must not observe completion
+ // until the encrypted result is ready, or release the browser lock.
+ const terminalStatus = run.status;
+ run.status = 'running';
+ const privateResult = await encryptPrivateResult(recipient, run.runId, raw);
+ if (run.status === 'aborting') {
+ run.status = 'aborted';
+ run.error = run.error || 'Aborted by cloud_abort.';
+ } else {
+ run.summary = PRIVATE_RESULT_PLACEHOLDER;
+ run.result = { type: PRIVATE_RESULT_TYPE, public_result: PRIVATE_RESULT_PLACEHOLDER,
+ private_result: privateResult };
+ run.status = terminalStatus;
+ }
+ }
+ }
} catch (error) {
run.pendingInput = null;
- run.status = run.status === 'aborting' ? 'aborted' : 'failed';
+ run.status = cancelled() || run.status === 'aborting' ? 'aborted' : 'failed';
- run.error = redactStrictTerminal(
+ run.error = recipient ? 'Private Cloud run failed before delivering its encrypted final answer. Review the redacted trace before retrying.' : redactStrictTerminal(
run, redactWorkflowValue(error?.message || String(error)), strictSecretMode,
);
run.finalUrl = cloudTerminalUrl(redactWorkflowValue(await getTabUrl(tabId)), { strictSecretMode });
diff --cc src/firefox/manifest.json
index 7b454a804,063f8740b..42ddae51a
--- a/src/firefox/manifest.json
+++ b/src/firefox/manifest.json
@@@ -36,40 -37,6 +37,40 @@@
}
},
"content_scripts": [
+ {
+ "matches": ["<all_urls>"],
+ "js": ["src/content/page-monitor.js"],
+ "run_at": "document_start",
+ "all_frames": true,
+ "match_about_blank": true,
+ "match_origin_as_fallback": true
+ },
+ {
+ "matches": ["<all_urls>"],
- "js": ["src/content/page-monitor-shadow.js", "src/content/captcha-callback-bridge.js"],
++ "js": ["src/content/page-monitor-shadow.js"],
+ "run_at": "document_start",
+ "world": "MAIN",
+ "all_frames": true,
+ "match_about_blank": true,
+ "match_origin_as_fallback": true
+ },
+ {
+ "matches": ["<all_urls>"],
+ "js": ["src/content/page-monitor-recovery.js", "src/content/page-monitor.js"],
+ "run_at": "document_end",
+ "all_frames": true,
+ "match_about_blank": true,
+ "match_origin_as_fallback": true
+ },
+ {
+ "matches": ["<all_urls>"],
+ "js": ["src/content/page-monitor-shadow.js"],
+ "run_at": "document_end",
+ "world": "MAIN",
+ "all_frames": true,
+ "match_about_blank": true,
+ "match_origin_as_fallback": true
+ },
{
"matches": ["<all_urls>"],
"js": ["src/content/file-picker-guard-loader.js"],
@@@ -151,7 -117,7 +152,8 @@@
"vendor/pdfjs/pdf.worker.mjs",
"src/agent/social-media-downloader.js",
"src/content/file-picker-guard-page.js",
+ "src/content/page-monitor-shadow.js",
+ "src/content/captcha-callback-bridge.js",
"skills/*"
],
"browser_specific_settings": {
diff --cc src/firefox/src/agent/agent.js
index ed75c4b92,7c0c80624..f3432826c
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@@ -12,12 -7,9 +12,13 @@@ import { redactSystemOneText, wrapSyste
import { createSystemOneJudge, isSystemOneResponseContractError, systemOneFailureReason, SYSTEM_ONE_COST_PROVIDER } from './systemone-judge.js';
import { firefoxBidi } from '../bidi/client.js';
import { SOCIAL_PLATFORMS, socialPublicationApiPlatform, normalizePublicationContract, publicationProgress, exactPublicationText, publicationMediaMatches, publicationContractMessages, publicationAuditMessages, publicationAuditAccepted } from './social-publish-contract.js';
+import { mastodonPublicationIntentMessages, normalizeMastodonPublicationIntent, captureNativeMastodonPublication, mastodonPublicationBinding, mastodonPublicationReceipt, mastodonPublicationNavigation } from './mastodon-publication.js';
import { AGENT_TOOLS, AGENT_TOOL_NAMES, RESERVED_AGENT_TOOL_NAMES, getToolsForMode, SYSTEM_PROMPT_ASK, SYSTEM_PROMPT_ACT, SYSTEM_PROMPT_ACT_COMPACT, SYSTEM_PROMPT_ACT_MID, SYSTEM_PROMPT_GENERATIVE_MEDIA, SYSTEM_PROMPT_DEV_APPENDIX } from './tools.js';
+ import { rejectedCompletionRecovery } from './completion-recovery.js';
import { validateToolArguments } from './tool-arguments.js';
+import { calculateExpression, calculationResultInText } from './calculator.js';
+import { isLiteralCalculationTask, literalCalculationSummaryFormatMatches } from './page-feedback-policy.js';
+import { chatRecoveryDisplayText, terminalRecoveryFinalText } from './chat-recovery-text.js';
import { isSessionQuotaError, serializeConversationForSession, SESSION_CONVERSATION_BUDGET_BYTES, SESSION_CONVERSATION_RETRY_BUDGET_BYTES } from './conversation-persistence.js';
import { formatErrorMessage } from '../error-format.js';
import { retryModelCall } from '../providers/model-retry.js';
@@@ -9463,29 -8705,49 +9471,70 @@@ export class Agent extends LoopDetecto
}
_repairToolCallArgs(name, args = {}, options = {}) {
+ if (name === 'read_page' && args && typeof args === 'object' && !Array.isArray(args)) {
- const next = { ...args };
++ let next = args;
+ let repaired = false;
- const boundWindow = (window) => {
- if (Number.isInteger(window.limit) && window.limit > 6000) {
- window.limit = 6000;
++ if (Number.isInteger(args.limit) && args.limit > 6000) {
++ next = { ...args, limit: 6000 };
++ repaired = true;
++ }
++ if (args.continuationArgs && typeof args.continuationArgs === 'object' && !Array.isArray(args.continuationArgs)) {
++ if (Number.isInteger(args.continuationArgs.limit) && args.continuationArgs.limit > 6000) {
++ next = { ...(repaired ? next : args), continuationArgs: { ...args.continuationArgs, limit: 6000 } };
+ repaired = true;
+ }
- };
- boundWindow(next);
- if (next.continuationArgs && typeof next.continuationArgs === 'object' && !Array.isArray(next.continuationArgs)) {
- next.continuationArgs = { ...next.continuationArgs };
- boundWindow(next.continuationArgs);
+ }
- return { args: next, repaired, note: repaired
- ? '[TOOL ARGUMENT REPAIR: Bounded read_page limit to its advertised maximum of 6000 characters. The requested offset and extraction options are preserved; continue with the exact returned continuationArgs when hasMore is true.]'
- : '' };
++ return {
++ args: repaired ? next : args,
++ repaired,
++ note: repaired
++ ? '[TOOL ARGUMENT REPAIR: Bounded read_page limit to its advertised maximum of 6000 characters. The requested offset and extraction options are preserved; continue with the exact returned continuationArgs when hasMore is true.]'
++ : '',
++ };
+ }
- if (name !== 'get_accessibility_tree' || !args || typeof args !== 'object' || Array.isArray(args)) {
+ if (!['get_accessibility_tree', 'wait_for_stable'].includes(name) || !args || typeof args !== 'object' || Array.isArray(args)) {
return { args, repaired: false, note: '' };
}
- const next = { ...args };
+ let next = { ...args };
let repaired = false;
+ const decimal = value => typeof value === 'string' && /^-?(?:0|[1-9]\d*)(?:\.\d+)?$/.test(value.trim())
+ && Number.isFinite(Number(value.trim())) ? Number(value.trim()) : value;
+ if (name === 'wait_for_stable') {
+ for (const key of ['quietMs', 'timeout']) {
+ const value = decimal(next[key]);
+ if (value !== next[key]) { next[key] = value; repaired = true; }
+ }
+ if (next.checkNetwork === 'true' || next.checkNetwork === 'false') {
+ next.checkNetwork = next.checkNetwork === 'true';
+ repaired = true;
+ }
+ return { args: next, repaired, note: repaired ? '[TOOL ARGUMENT REPAIR: Normalized declared wait_for_stable scalar types. Schema validation still applies.]' : '' };
+ }
+ if (Object.hasOwn(next, 'continuationArgs')) {
+ let continuation = next.continuationArgs;
+ if (typeof continuation === 'string') {
+ try { continuation = continuation.length <= 32768 ? JSON.parse(continuation) : null; } catch { continuation = null; }
+ }
+ const allowed = ['filter', 'maxDepth', 'maxChars', 'ref_id', 'page', 'tree_revision'];
+ const valid = continuation && typeof continuation === 'object' && !Array.isArray(continuation)
+ && Object.keys(continuation).length > 0
+ && Object.keys(continuation).every(key => allowed.includes(key))
+ && Object.entries(continuation).every(([key, value]) => !Object.hasOwn(next, key) || Object.is(next[key], value));
+ if (!valid) {
+ return { args, repaired: false, note: '', result: {
+ success: false, invalidArguments: true, invalidToolArguments: true,
+ noDispatch: true, dispatched: false, errorCode: 'invalid_tool_arguments',
+ invalidArgumentNames: ['$.continuationArgs'],
+ error: 'get_accessibility_tree continuationArgs must be one complete JSON object containing only declared read fields, with no conflicting top-level fields. Re-emit the exact returned continuation fields as top-level arguments.',
+ } };
+ }
+ next = this._normalizeContinuationToolArgs(name, { ...next, continuationArgs: continuation });
+ repaired = true;
+ }
+ for (const key of ['maxDepth', 'maxChars', 'page']) {
+ const value = decimal(next[key]);
+ if (value !== next[key]) { next[key] = value; repaired = true; }
+ }
const filter = String(next.filter || '').trim();
if (filter && !/^(?:all|visible|interactive)$/i.test(filter)) {
const match = filter.match(/\b(all|visible|interactive)\b/i);
@@@ -39330,8 -36787,10 +39439,12 @@@ Rules: no prose intro, no conclusion, n
if (completionRecoveryPolicy) {
tools = completionRecoveryPolicy.tools;
}
+ const invalidArgumentToolChoice = !invalidArgumentRecoveryUsed && !completionRecoveryPolicy
+ ? this._invalidArgumentRecoveryToolChoice(provider, messages, tools) : null;
- const completionToolChoice = completionRecoveryPolicy?.toolChoice || invalidArgumentToolChoice;
+ const completionToolChoice = completionRecoveryPolicy?.toolChoice
- || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null) || null;
++ || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null)
++ || invalidArgumentToolChoice || null;
+ if (invalidArgumentToolChoice) invalidArgumentRecoveryUsed = true;
allowedToolNames = new Set(tools.map(t => t.function.name));
toolSchemas = new Map(tools.map(t => [t.function.name, t.function.parameters]));
@@@ -40432,8 -37847,10 +40552,12 @@@
if (completionRecoveryPolicy) {
tools = completionRecoveryPolicy.tools;
}
+ const invalidArgumentToolChoice = !invalidArgumentRecoveryUsed && !completionRecoveryPolicy
+ ? this._invalidArgumentRecoveryToolChoice(provider, messages, tools) : null;
- const completionToolChoice = completionRecoveryPolicy?.toolChoice || invalidArgumentToolChoice;
+ const completionToolChoice = completionRecoveryPolicy?.toolChoice
- || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null) || null;
++ || (provider.supportsTools ? calculationToolPolicy?.toolChoice : null)
++ || invalidArgumentToolChoice || null;
+ if (invalidArgumentToolChoice) invalidArgumentRecoveryUsed = true;
allowedToolNames = new Set(tools.map(t => t.function.name));
toolSchemas = new Map(tools.map(t => [t.function.name, t.function.parameters]));
@@@ -40466,8 -37882,8 +40590,9 @@@
let reasoningContent = '';
let streamUsage = null;
let finishReason = '';
+ let rejectedToolResponse = null;
+ await this._capturePageFeedbackModelState(tabId, pendingVisionFallbackMessages || messages);
const fastResult = await this._maybeJevFastTurn(tabId, userMessage, messages, mode, allowedToolNames, provider, costState, runOptions, completionRecoveryPolicy);
const streamOpts = this._cloudGenerationOptions(provider, {
signal: this._runAbortSignal(tabId),
diff --cc src/firefox/src/agent/tool-arguments.js
index 788549254,a7f300726..8ae52dac9
--- a/src/firefox/src/agent/tool-arguments.js
+++ b/src/firefox/src/agent/tool-arguments.js
@@@ -186,16 -181,11 +186,19 @@@ export function validateToolArguments(t
const failures = [];
validateValue(normalizedArgs, closedParameters, '$', failures);
if (failures.length) {
- return validationFailure(toolName, failures, `Invalid or undeclared argument(s): ${[...new Set(failures)].join(', ')}.`);
+ const enumGuidance = Object.entries(closedParameters.properties || {})
+ .filter(([key, schema]) => failures.includes(`$.${key}`) && Array.isArray(schema.enum))
+ .map(([key, schema]) => `${key} must be one of ${JSON.stringify(schema.enum)}.`).join(' ');
+ return validationFailure(toolName, failures, `Invalid or undeclared argument(s): ${[...new Set(failures)].join(', ')}. ${enumGuidance}`.trim());
}
+ if (toolName === 'get_accessibility_tree') {
+ const refs = [['$.ref_id', normalizedArgs.ref_id],
+ ['$.continuationArgs.ref_id', normalizedArgs.continuationArgs?.ref_id]];
+ const invalid = refs.filter(([, value]) => typeof value === 'string'
+ && AX_REF_ID_REGEX.exec(value)?.[0] !== value).map(([path]) => path);
+ if (invalid.length) return validationFailure(toolName, invalid,
+ 'Use an exact observed native ref_id (ref_ followed by digits), or omit ref_id for a root read. Do not append JSON punctuation or invent an anchor.');
+ }
if (toolName === 'click') {
const clickFailure = validateClickTarget(normalizedArgs);
if (clickFailure) return clickFailure;
diff --cc src/firefox/src/cloud-runs.js
index 7acfd4f7f,78cc5f955..d6d7ec650
--- a/src/firefox/src/cloud-runs.js
+++ b/src/firefox/src/cloud-runs.js
@@@ -1081,8 -1071,10 +1092,10 @@@ export function createCloudRunControlle
finally { if (key != null) startingConversations.delete(key); }
}
- async function startRunReserved(msg = {}) {
+ async function startRunReserved(msg = {}, startup = null) {
await hydrate();
+ const recipient = msg.private_result_public_key == null
+ ? null : await importPrivateResultRecipient(msg.private_result_public_key);
const suppliedRunId = msg.runId ?? msg.run_id;
const requestedRunId = suppliedRunId == null ? '' : String(suppliedRunId).trim();
const parentRunId = String(msg.parentRunId || msg.parent_run_id || '').trim() || null;
@@@ -1195,22 -1199,12 +1227,24 @@@
completedAt: null,
};
runs.set(run.runId, run);
+ startingRunIds.delete(runId);
+ if (recipient) privateRecipients.set(run.runId, recipient);
+ const runController = new AbortController();
+ runAbortControllers.set(runId, runController);
+ const cancelled = () => runController.signal.aborted;
+ const checkCancellation = () => {
+ if (cancelled()) throw new Error('Abort requested.');
+ };
+ const cancellationOptions = {
+ signal: runController.signal,
+ isDetachedStartCancelled: cancelled,
+ };
try {
await persist();
} catch (error) {
runs.delete(run.runId);
+ runAbortControllers.delete(runId);
+ privateRecipients.delete(run.runId);
startingTabs.delete(tabId);
throw error;
}
@@@ -1408,10 -1398,29 +1444,29 @@@
}
}
}
+ if (recipient && run.status !== 'aborted') {
+ const raw = structured ? privateStructuredResults.get(run.runId) : content;
+ if (raw !== undefined) {
+ // Encryption is asynchronous. Pollers must not observe completion
+ // until the encrypted result is ready, or release the browser lock.
+ const terminalStatus = run.status;
+ run.status = 'running';
+ const privateResult = await encryptPrivateResult(recipient, run.runId, raw);
+ if (run.status === 'aborting') {
+ run.status = 'aborted';
+ run.error = run.error || 'Aborted by cloud_abort.';
+ } else {
+ run.summary = PRIVATE_RESULT_PLACEHOLDER;
+ run.result = { type: PRIVATE_RESULT_TYPE, public_result: PRIVATE_RESULT_PLACEHOLDER,
+ private_result: privateResult };
+ run.status = terminalStatus;
+ }
+ }
+ }
} catch (error) {
run.pendingInput = null;
- run.status = run.status === 'aborting' ? 'aborted' : 'failed';
+ run.status = cancelled() || run.status === 'aborting' ? 'aborted' : 'failed';
- run.error = redactStrictTerminal(
+ run.error = recipient ? 'Private Cloud run failed before delivering its encrypted final answer. Review the redacted trace before retrying.' : redactStrictTerminal(
run, redactWorkflowValue(error?.message || String(error)), strictSecretMode,
);
run.finalUrl = cloudTerminalUrl(redactWorkflowValue(await getTabUrl(tabId)), { strictSecretMode });