Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Bind approved X DM bodies in workflow metadata
Emre Sokulluf2548518f7e9
diff --git a/docs/social-publication-contract.md b/docs/social-publication-contract.mdindex afa3dd550..8852263bc 100644--- a/docs/social-publication-contract.md+++ b/docs/social-publication-contract.md@@ -59,4 +59,4 @@ The config accepts the existing provider fields (`providerName`, `baseUrl`, `modCalls run sequentially. Reports distinguish first-response validity, repairs, semantic contract errors, audit false accepts/rejects, token usage, and median/p95 latency. First-call latency is reported separately without assuming a cold server. Default results are ignored local artifacts under `test/llm/results-social-publication/`, with a Markdown report, per-case JSONL, and prompt/fixture hashes. The runner never invokes a browser action or publishes a post.-X direct messages use their own message workflow, not the public publication contract. On `/i/chat/<conversation>`, the conversation header binds the recipient. An exact display-name or bare-username match resolves to that header's canonical handle during planning or before dispatch; subsequent verification requires the same handle, never a replacement account with the same display name. A send binds the exact body and the identities of all mounted message rows, including hidden and pending ones. Completion requires a new matching outgoing row after the prior conversation tail with `data-send-status="sent"`, an empty composer, and the same conversation. The open X chat composer does not override that delivery evidence; other adapters retain their dialog checks. Workflow field requirements are loaded even when repeated-item tracking is disabled. Incoming, pending, failed, or pre-existing messages cannot satisfy it.+X direct messages use their own message workflow, not the public publication contract. On `/i/chat/<conversation>`, the conversation header binds the recipient. An exact display-name or bare-username match resolves to that header's canonical handle during planning or before dispatch; subsequent verification requires the same handle, never a replacement account with the same display name. A send binds the exact body and the identities of all mounted message rows, including hidden and pending ones. An explicitly supplied body is required by the message-field classifier and is recovered from the approved task when a classifier response omits it. Completion requires a new matching outgoing row after the prior conversation tail with `data-send-status="sent"`, an empty composer, and the same conversation. The open X chat composer does not override that delivery evidence; other adapters retain their dialog checks. Workflow field requirements are loaded even when repeated-item tracking is disabled. Incoming, pending, failed, or pre-existing messages cannot satisfy it.diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 259ec10f8..9af61b8e6 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -28399,7 +28399,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d'Exception: for siteContext.workflow.job="upload-release-assets" with requiresLedger=true, use mode=active and list every concrete requested target even when there is exactly one. Copy each exact requested filename or path into targets; do not merge or omit assets. When the user names the release tag, also return it as workflowFields=[{"field":"tag","value":"exact tag"}]; return workflowFields=[] when no tag is named.','For siteContext.workflow.job="update-metadata", workflowFields must contain every metadata field explicitly requested by the user and its complete exact intended value. Use canonical field names title, description, visibility, audience, tags, category, playlist, language, license, comments, embedding, paid_promotion, recording_date, or recording_location. Never infer a field or value from page content.','For siteContext.workflow.job="publish-release", "publish-post", "publish-content", or "edit-file-and-commit", workflowFields must contain every publication field explicitly requested by the user and its complete exact intended value (for long post bodies or notes exceeding response budget, provide a short excerpt; complete bodies are preserved from task context). Use canonical field names tag, title, notes, body, visibility, attachment, path, branch, or commit_message; for publish-post only, also use account when the user explicitly names the publishing account and alt_text when the user explicitly requests attachment alternative text. When different attachments have different alt text, return one alt_text entry per attachment and set its attachment property to that exact filename. For edit-file-and-commit, include path, branch, and commit_message only when the user explicitly supplied them; the runtime separately binds the exact verified editor content. Never infer a field or value from page content.',- 'For siteContext.workflow.job="draft-email" or "send-email", workflowFields must contain every message field explicitly requested by the user and its complete exact intended value. Use canonical field names subject or body. Never infer a field or value from page content.',+ 'For siteContext.workflow.template="message", workflowFields must contain every message field explicitly requested by the user and its complete exact intended value. Use canonical field names subject or body. A direct-message body explicitly supplied by the user must be returned as body, including for send-message. Never infer a field or value from page content.','For siteContext.workflow.template="transaction", workflowFields must contain every booking detail explicitly requested by the user and its exact value. Use canonical field names train, travel_date, departure, arrival, passenger, or seat_class. Never infer a detail from page content.','For siteContext.workflow.template="form", workflowLabelValues must contain one entry per field the user supplied an exact value for, as {"label":"the field in the user\'s words","value":"the exact value"}. Return workflowLabelValues=[] when the user supplied no exact values, and never copy a value from page content.','For siteContext.workflow.template="form", workflowRequiredLabels must list every form question, field, upload, or attachment the user explicitly asked to provide, copied from the user request in the user\'s own words. A field the page marks optional still belongs in this list when the user asked for it. Return workflowRequiredLabels=[] when the user named none, and never infer one from page content.',@@ -28470,7 +28470,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dif (this._workflowExtractedBodySupersedesClassified(bodyReq.value, extractedBody)) {bodyReq.value = this._workflowMetadataValue(extractedBody, 25000);}- } else if (siteWorkflow?.job?.id === 'publish-post' || siteWorkflow?.job?.id === 'publish-content') {+ } else if (siteWorkflow?.job?.template === 'message'+ || siteWorkflow?.job?.id === 'publish-post' || siteWorkflow?.job?.id === 'publish-content') {details.items.push({field: 'body',value: this._workflowMetadataValue(extractedBody, 25000),diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex 6f3a45a7a..7bba34188 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -26013,7 +26013,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d'Exception: for siteContext.workflow.job="upload-release-assets" with requiresLedger=true, use mode=active and list every concrete requested target even when there is exactly one. Copy each exact requested filename or path into targets; do not merge or omit assets. When the user names the release tag, also return it as workflowFields=[{"field":"tag","value":"exact tag"}]; return workflowFields=[] when no tag is named.','For siteContext.workflow.job="update-metadata", workflowFields must contain every metadata field explicitly requested by the user and its complete exact intended value. Use canonical field names title, description, visibility, audience, tags, category, playlist, language, license, comments, embedding, paid_promotion, recording_date, or recording_location. Never infer a field or value from page content.','For siteContext.workflow.job="publish-release", "publish-post", "publish-content", or "edit-file-and-commit", workflowFields must contain every publication field explicitly requested by the user and its complete exact intended value (for long post bodies or notes exceeding response budget, provide a short excerpt; complete bodies are preserved from task context). Use canonical field names tag, title, notes, body, visibility, attachment, path, branch, or commit_message; for publish-post only, also use account when the user explicitly names the publishing account and alt_text when the user explicitly requests attachment alternative text. When different attachments have different alt text, return one alt_text entry per attachment and set its attachment property to that exact filename. For edit-file-and-commit, include path, branch, and commit_message only when the user explicitly supplied them; the runtime separately binds the exact verified editor content. Never infer a field or value from page content.',- 'For siteContext.workflow.job="draft-email" or "send-email", workflowFields must contain every message field explicitly requested by the user and its complete exact intended value. Use canonical field names subject or body. Never infer a field or value from page content.',+ 'For siteContext.workflow.template="message", workflowFields must contain every message field explicitly requested by the user and its complete exact intended value. Use canonical field names subject or body. A direct-message body explicitly supplied by the user must be returned as body, including for send-message. Never infer a field or value from page content.','For siteContext.workflow.template="transaction", workflowFields must contain every booking detail explicitly requested by the user and its exact value. Use canonical field names train, travel_date, departure, arrival, passenger, or seat_class. Never infer a detail from page content.','For siteContext.workflow.template="form", workflowLabelValues must contain one entry per field the user supplied an exact value for, as {"label":"the field in the user\'s words","value":"the exact value"}. Return workflowLabelValues=[] when the user supplied no exact values, and never copy a value from page content.','For siteContext.workflow.template="form", workflowRequiredLabels must list every form question, field, upload, or attachment the user explicitly asked to provide, copied from the user request in the user\'s own words. A field the page marks optional still belongs in this list when the user asked for it. Return workflowRequiredLabels=[] when the user named none, and never infer one from page content.',@@ -26084,7 +26084,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dif (this._workflowExtractedBodySupersedesClassified(bodyReq.value, extractedBody)) {bodyReq.value = this._workflowMetadataValue(extractedBody, 25000);}- } else if (siteWorkflow?.job?.id === 'publish-post' || siteWorkflow?.job?.id === 'publish-content') {+ } else if (siteWorkflow?.job?.template === 'message'+ || siteWorkflow?.job?.id === 'publish-post' || siteWorkflow?.job?.id === 'publish-content') {details.items.push({field: 'body',value: this._workflowMetadataValue(extractedBody, 25000),diff --git a/test/run.js b/test/run.jsindex cf8425b64..a0107e319 100644--- a/test/run.js+++ b/test/run.js@@ -93884,6 +93884,73 @@ test('publication workflows classify and bind requested payload fields', async (assert.match(prompt, /\bcanonical field names tag, title, notes, body, visibility, attachment, path, branch, or commit_message\b/);assert.match(prompt, /for publish-post only, also use account/);assert.match(prompt, /For edit-file-and-commit, include path, branch, and commit_message only when the user explicitly supplied them/);++ const xTabId = 8987 + index;+ const xUrl = 'https://x.com/i/chat/123-456';+ const xTask = 'Send Alex this exact message on X: "The release is ready."';+ const xWorkflow = agent._resolvePlannerSiteWorkflow(xUrl, {+ request_kind: 'execute',+ requires_submission: true,+ messaging: { target_kind: 'named', recipients: ['@altryne'] },+ });+ assert.equal(xWorkflow?.job?.id, 'send-message');+ agent.conversations.set(xTabId, [+ { role: 'system', content: 'system' },+ { role: 'user', content: xTask },+ ]);+ const xGuard = agent._startPlanExecutionGuard(xTabId, 'act', {+ requestKind: 'execute',+ requiresStateChange: true,+ requiresSubmission: true,+ messaging: { target_kind: 'named', recipients: ['@altryne'] },+ siteWorkflow: xWorkflow,+ });+ let xPrompt = '';+ agent._chatWithCostAllowance = async (_provider, messages) => {+ xPrompt = messages[0].content;+ // This was previously a compliant classifier response for send-message.+ return { content: JSON.stringify({+ mode: 'inactive', allowedActions: [], forbiddenActions: [], targets: [],+ workflowFields: [], confidence: 0.99, pageScopePolicy: 'page',+ }) };+ };+ await agent._ensureProgressSessionForCurrentTask(xTabId, {+ provider: { chat: async () => ({ content: '{}' }) },+ progressLedgerPolicy: 'disabled',+ taskText: xTask,+ pageScope: xUrl,+ });+ assert.match(xPrompt, /siteContext\.workflow\.template="message"/,+ `${AgentClass.name}: the classifier did not require direct-message fields`);+ assert.deepEqual(xGuard.workflowMetadataRequirements, [+ { field: 'body', value: 'The release is ready.' },+ ], `${AgentClass.name}: an explicit X DM body was not bound after an empty classifier response`);+ const xTerminal = (body) => {+ const workflowBinding = agent._workflowSubmitBindingForAttempt(xTabId, xUrl, {+ messageRecipientGuardRequired: true,+ messageRecipientDispatchBinding: { token: `x-body-${body}` },+ messageRecipientBody: body,+ messageRecipientBodyBaselineCount: 0,+ messageRecipientExistingMessageIds: [],+ });+ return agent._workflowTerminalEvidenceFromDone(xTabId, { liveRegionMessages: [] }, xUrl, {+ submit: { dispatched: true, observedAfterSubmit: true, originatingUrl: xUrl, workflowBinding },+ verifiedFinalSubmit: false,+ relevantForms: 1,+ }, {+ success: true,+ conclusive: true,+ composerEmpty: true,+ strongRecipientCandidates: [{ identity: '@altryne', role: 'to' }],+ existingMessageIds: ['message-new'],+ matchingOutgoingMessageIds: ['message-new'],+ matchingOutgoingMessageCount: 1,+ });+ };+ assert.equal(xTerminal('Changed message'), null,+ `${AgentClass.name}: a sent X DM drifted from the approved body`);+ assert.equal(xTerminal('The release is ready.')?.verificationKind, 'message_sent',+ `${AgentClass.name}: the approved X DM body could not complete`);}});