Open-source AI browser agent for Chrome and Firefox (monorepo) đź§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Remove obsolete submission-authorization caveat from report and handoff docs
Emre Sokullu984587830894
diff --git a/test/llm-tiny/runpod/handoff/HANDOFF.md b/test/llm-tiny/runpod/handoff/HANDOFF.mdindex c1e9e6ac8..96c118685 100644--- a/test/llm-tiny/runpod/handoff/HANDOFF.md+++ b/test/llm-tiny/runpod/handoff/HANDOFF.md@@ -115,7 +115,7 @@ Our local Node guard was a laptop process making DELETE calls at an absolute tim## Known interpretation limitations: preserve, do not silently repair- Conference planning wording mixes base/daily dollars and a fee per session; the frozen key charges the base once. Different plausible readings explain some failed totals.-- Evidence tasks request decision/source ID but not explicitly form submission; the scorer requires submission. Fara's native authorization caution can interact with this mismatch. Source ID is effectively a document title, whereas visible controls also have numeric indices.+- Evidence tasks explicitly authorize submitting the form to record the decision, as the scorer requires. Source ID is effectively a document title, whereas visible controls also have numeric indices.- Fara 4B workflow strict failures were `Draft` vs exact `draft` only, with all other checks passing. Do not call them multi-step reasoning failures.- Browser Use workflow-01 omitted Brief evidence and invented values. Fara planning-01 selected a more expensive option; evidence-01 misread source facts. These are separate observed model errors.- Fara browser Find shortcuts did not open Find in headless Mac Chromium; typing stayed in the page input. This adapter/browser-surface limitation produced no thrown key-dispatch error. Do not silently add unsupported browser chrome or change its frozen adapter.diff --git a/test/llm-tiny/runpod/v2-report.mjs b/test/llm-tiny/runpod/v2-report.mjsindex 9ce98cb19..d75e0ec64 100644--- a/test/llm-tiny/runpod/v2-report.mjs+++ b/test/llm-tiny/runpod/v2-report.mjs@@ -38,7 +38,7 @@ else {for (const cat of ['planning','calculation','evidence','workflow','recovery','visual']) md += `| ${cat} | ${names.map(n=>{const c=summaries.find(s=>s.name===n)?.categories[cat];return c?`${c.passed}/${c.cases} (${(100*c.successRate).toFixed(0)}%)`:'—';}).join(' | ')} |\n`;}md += `\n## Original shared $30 budget\n\n- Conservative RunPod compute/storage accrued: ${money(computeAccrued)}. Includes every download, setup, inference, idle and cleanup second from creation until deletion, including earlier v1 hosting. The $1.65/hour budget rate exceeds the $1.59/hour GPU quote and includes a storage allowance.\n- All recorded API charges plus unresolved request reservations: ${money(budget.apiChargedOrReservedUsd)}. This includes $0.029319759 incurred before v2 and $0.031821054 for the matching DeepSeek pilot.\n- Total conservative accrued: ${money(computeAccrued+budget.apiChargedOrReservedUsd)}. Remaining: ${money(remaining)}.\n- Active resource/runtime commitments: ${money(computeThroughExpiry-computeAccrued)}; cleanup reserve ${money(budget.cleanupReserveUsd)}; future deployment planning reserve ${money(budget.futureComputeReserveUsd)}. ${cleanup?'All future reservations were released after confirmed cleanup. No requests are pending; the $2 API policy allowance is not an active obligation.':''}\n- CPU setup, new storage and cloud retention incurred under this handoff: $0. No resource was provisioned.\n- ${cleanup?'RunPod deletion and an empty live Pod inventory are independently confirmed. Its billing feed had not settled; an empty billing response is not zero spend.':'Existing deployment expiries are unchanged.'} These are conservative estimates, not a settled invoice or a provider-enforced spending cap.\n- GPU costs cover a shared lifetime across model downloads and both suite versions, not an attributable per-model inference invoice. No unsupported per-model allocation is invented.\n`;-md += '\n## Pilot interpretation caveats\n\nThe conference-space task uses the heading “Base/daily dollars” and wording that can be read as a fee per session or both amounts per session, while the frozen answer charges the base once. This is a task-wording ambiguity, not clean evidence of an arithmetic deficit. Evidence tasks request a “source ID” but label documents only by title; numeric control indices are not accepted by the exact-string grader. Their task text asks to decide and give the source without explicitly authorizing form submission, while the scorer requires it. This interacts with the native Fara safety prompt, which requires authorization for irreversible submissions. Answer-only termination and wrong submitted values must be distinguished from broken action execution. Preserve the frozen strict scores and flag these ambiguities; do not silently change answers, drop tasks, or re-freeze.\n';+md += '\n## Pilot interpretation caveats\n\nThe conference-space task uses the heading “Base/daily dollars” and wording that can be read as a fee per session or both amounts per session, while the frozen answer charges the base once. This is a task-wording ambiguity, not clean evidence of an arithmetic deficit. Evidence tasks request a “source ID” but label documents only by title; numeric control indices are not accepted by the exact-string grader. Their task text explicitly authorizes submitting the form to record the decision, as the scorer requires. Answer-only termination and wrong submitted values must be distinguished from broken action execution. Preserve the frozen strict scores and flag these ambiguities; do not silently change answers, drop tasks, or re-freeze.\n';md += '\nThe two Fara 4B workflow failures are exact-case formatting failures: both completed every other check (5/6 each) but submitted `Draft` where the frozen answer is `draft`. These remain strict failures; they are not failures to calculate totals or follow the multi-stage workflow.\n';md += '\n## Operational failures\n\n';for (const s of summaries) {