Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
humanized
Emre Sokullu1a80254e6dfb
diff --git a/web/blog/index.html b/web/blog/index.htmlindex 6d47abd05..27e56ed93 100644--- a/web/blog/index.html+++ b/web/blog/index.html@@ -674,7 +674,7 @@"headline": "Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain","url": "https://webbrain.one/blog/webbrain-compass-tiny-v1","datePublished": "2026-09-05",- "description": "WebBrain Compass Tiny v1 is a private 2.6B WebBrain Compact-model preview. In our first routing suite it ties Qwen3.6-35B-A3B on tool-name conformity, improves the same local LFM2.5 base model by eight routing outcomes, and keeps the model small enough for edge-oriented deployment.",+ "description": "WebBrain Compass Tiny v1 is a private preview of a 2.6B model built for WebBrain's Compact tool loop. In our first routing suite it matches Qwen3.6-35B-A3B on tool-name conformity and beats the local LFM2.5 base model it was trained from by eight routing outcomes, at a size meant for edge deployment.","author": {"@type": "Person","name": "Emre Sokullu",@@ -1319,10 +1319,10 @@</section><div class="post-list" id="post-list">- <a href="/blog/webbrain-compass-tiny-v1" class="post-card" data-search="Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain A 2.6B model is not a 35B general-reasoning replacement. But in the narrow, production-shaped WebBrain Compact routing loop, Compass Tiny v1 reaches the same tool-name conformity as Qwen3.6-35B-A3B and improves its matched local base model by eight routing outcomes. WebBrain Compass Tiny v1 is a private 2.6B WebBrain Compact-model preview. In our first routing suite it ties Qwen3.6-35B-A3B on tool-name conformity, improves the same local LFM2.5 base model by eight routing outcomes, and keeps the model small enough for edge-oriented deployment. September 5, 2026 2026-09-05 WebBrain Compass Tiny browser agent tool calling compact mode edge AI LFM2.5 Qwen">+ <a href="/blog/webbrain-compass-tiny-v1" class="post-card" data-search="Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain A 2.6B model does not replace a 35B one at general reasoning. But inside WebBrain's narrow, production-shaped Compact routing loop, Compass Tiny v1 hits the same tool-name conformity as Qwen3.6-35B-A3B and beats its matched local base model by eight routing outcomes. WebBrain Compass Tiny v1 is a private preview of a 2.6B model built for WebBrain's Compact tool loop. In our first routing suite it matches Qwen3.6-35B-A3B on tool-name conformity and beats the local LFM2.5 base model it was trained from by eight routing outcomes, at a size meant for edge deployment. September 5, 2026 2026-09-05 WebBrain Compass Tiny browser agent tool calling compact mode edge AI LFM2.5 Qwen"><div class="post-meta">September 5, 2026 · 6 min read</div><div class="post-title">Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain</div>- <div class="post-excerpt">A 2.6B model is not a 35B general-reasoning replacement. But in the narrow, production-shaped WebBrain Compact routing loop, Compass Tiny v1 reaches the same tool-name conformity as Qwen3.6-35B-A3B and improves its matched local base model by eight routing outcomes.</div>+ <div class="post-excerpt">A 2.6B model does not replace a 35B one at general reasoning. But inside WebBrain's narrow, production-shaped Compact routing loop, Compass Tiny v1 hits the same tool-name conformity as Qwen3.6-35B-A3B and beats its matched local base model by eight routing outcomes.</div></a><a href="/blog/deepseek-v4-flash-vision-openrouter" class="post-card" data-search="DeepSeek V4 Flash Vision Exp tested: a great planner with experimental eyes DeepSeek V4 Flash Vision Exp tested: a great planner with experimental eyes DeepSeek V4 Flash Vision Exp delivers the best first-action routing of our three new Flash models and a 2.60-second planner median. Its vision side is fast, but only 68 of 100 screenshots pass the strict rubric. We tested DeepSeek V4 Flash Vision Exp on 200 WebBrain browser-agent cases. Its text planning beats Qwen3.8 Flash and GLM-5.3 Flash, but its experimental vision route trails both. August 27, 2026 2026-08-27 DeepSeek V4 Flash Vision Exp DeepSeek V4 Flash 0731 OpenRouter multimodal model benchmark browser agent mixture of experts vision language model tool calling AI API pricing WebBrain"><div class="post-meta">August 27, 2026 · 9 min read</div>diff --git a/web/blog/posts/webbrain-compass-tiny-v1.md b/web/blog/posts/webbrain-compass-tiny-v1.mdindex 9313147c3..ce41768ac 100644--- a/web/blog/posts/webbrain-compass-tiny-v1.md+++ b/web/blog/posts/webbrain-compass-tiny-v1.md@@ -6,19 +6,19 @@ sortOrder: -281date: 2026-09-05readTime: 6 min readdescription: >- WebBrain Compass Tiny v1 is a private 2.6B WebBrain Compact-model preview. In our first routing suite it ties Qwen3.6-35B-A3B on tool-name conformity, improves the same local LFM2.5 base model by eight routing outcomes, and keeps the model small enough for edge-oriented deployment.+ WebBrain Compass Tiny v1 is a private preview of a 2.6B model built for WebBrain's Compact tool loop. In our first routing suite it matches Qwen3.6-35B-A3B on tool-name conformity and beats the local LFM2.5 base model it was trained from by eight routing outcomes, at a size meant for edge deployment.excerpt: >- A 2.6B model is not a 35B general-reasoning replacement. But in the narrow, production-shaped WebBrain Compact routing loop, Compass Tiny v1 reaches the same tool-name conformity as Qwen3.6-35B-A3B and improves its matched local base model by eight routing outcomes.+ A 2.6B model does not replace a 35B one at general reasoning. But inside WebBrain's narrow, production-shaped Compact routing loop, Compass Tiny v1 hits the same tool-name conformity as Qwen3.6-35B-A3B and beats its matched local base model by eight routing outcomes.titleTag: >Introducing WebBrain Compass Tiny v1 - WebBrain BlogogTitle: >WebBrain Compass Tiny v1: Compact browser-tool routing at the edgeogDescription: >- A first look at WebBrain's 2.6B Compact model preview, its tool-routing benchmark, and the road toward open weights.+ A first look at WebBrain's 2.6B Compact model preview, its tool-routing benchmark, and what an open-weights release would take.twitterTitle: >WebBrain Compass Tiny v1: an edge-sized Compact modeltwitterDescription: >- Compass Tiny v1 ties a 35B Qwen model on WebBrain Compact tool-name routing. More tests and v1.1/v2 work are coming.+ Compass Tiny v1 matches a 35B Qwen model on WebBrain Compact tool-name routing. More tests and v1.1/v2 work are coming.keywords:- WebBrain Compass Tiny- browser agent@@ -30,12 +30,12 @@ keywords:author: Emre SokulluauthorUrl: https://emresokullu.comlede: >- **WebBrain Compass Tiny v1 is our first edge-sized model built specifically for the Compact browser-tool loop.** It is not a claim that 2.6B parameters equal a 35B model at general reasoning. It is evidence that a focused, grounded routing policy can make a small model useful inside a real browser agent: in this first suite, Compass ties Qwen3.6-35B-A3B on the expected tool name and improves the matched local LFM2.5-2.6B base model by eight routing outcomes. Qwen3.8-27B remains the quality leader when its cost and footprint fit the deployment.+ **WebBrain Compass Tiny v1 is our first edge-sized model built for the Compact browser-tool loop.** We are not claiming that 2.6B parameters match a 35B model at general reasoning. What this first suite shows is narrower: a focused, grounded routing policy can make a small model useful inside a real browser agent. Compass matches Qwen3.6-35B-A3B at picking the expected tool name, and it beats the LFM2.5-2.6B base model it was trained from by eight routing outcomes. Qwen3.8-27B is still the quality leader wherever its cost and footprint fit.---## The short version-Compass Tiny v1 is a **private preview** today. We are preparing an open-weights release so it can be downloaded and run locally, subject to the remaining release, provenance, and packaging checks. Until that work is complete, it is not a public download and this article should not be read as a release announcement.+Compass Tiny v1 is a private preview today. We are preparing an open-weights release so it can be downloaded and run locally, once the remaining release, provenance, and packaging checks are done. Until that work is finished it is not a public download, and this article is not a release announcement.The model is tuned for three Compact behaviors in WebBrain:@@ -43,18 +43,15 @@ The model is tuned for three Compact behaviors in WebBrain:2. Emit a grounded Compact tool call when the page context supports it.3. Abstain or escalate when direct execution would be unjustified.-That is deliberately narrower than “be a universally smarter model.” It is the right narrowness for a browser agent that must choose a next action from a constrained toolset.+That is narrower than "be a universally smarter model," and the narrowness is the point. A browser agent has to choose its next action from a constrained toolset.## What we measured-We ran the current Compact Chrome suite: 100 first-action cases and 100 stateful scenarios. Eleven scenarios are intentionally out of scope for Compact and are excluded from the scenario-quality denominator, leaving 89 scored scenarios.+We ran the current Compact Chrome suite: 100 first-action cases and 100 stateful scenarios. Eleven of the scenarios are intentionally out of scope for Compact, so they are excluded from the scenario-quality denominator. That leaves 89 scored scenarios.-The table uses two different signals:+The table reports two different signals. Exact reference action means the tool and the arguments matched the canonical next action. Expected tool name credits the model for reaching for the right tool even when the arguments differ, which measures routing rather than a completed browser task.-- **Exact reference action** means the tool and arguments matched the canonical next action.-- **Expected tool name** credits the expected tool even when arguments differ. It is useful for measuring routing, but it is not a completed browser task.--The exact reference is a Sonnet-authored canonical next action. It is a regression-oriented reference, not an oracle for every reasonable live-browser strategy: a careful model may inspect before a direct click, for example. That distinction matters here.+The canonical next action was authored by Sonnet. It works as a regression reference, not as an oracle for every reasonable live-browser strategy: a careful model might inspect something before clicking it directly. Keep that in mind when reading the exact-reference column.| Model | Parameters | First-turn structured calls | Exact reference action (of 89) | Expected tool name (of 89) || --- | ---: | ---: | ---: | ---: |@@ -64,38 +61,32 @@ The exact reference is a Sonnet-authored canonical next action. It is a regressi| LFM2.5-2.6B base (local) | 2.6B | 80 / 100 | 4 | 34 || Qwen3.5-9B | 9B | 84 / 100 | 2 | 12 |-Compass therefore does not beat the 35B model on the strict reference. It **matches it on tool-name routing** in this Compact suite. It also exceeds the 9B Qwen result by 30 expected-tool-name outcomes and the matched local LFM base model by eight. Those are promising results for a model intended to live near the edge of the product, where size, VRAM, latency, and controllability matter as much as raw general capability.+So Compass does not beat the 35B model on the strict reference. It matches it on tool-name routing in this Compact suite. It also lands 30 expected-tool-name outcomes above the 9B Qwen and eight above the local LFM base model. For a model meant to run near the edge of the product, where size, VRAM, latency, and controllability count for as much as raw general capability, that is a good starting position.## The controlled base-model comparison-The base-model row is the comparison we care about most. We served the pinned `LiquidAI/LFM2.5-2.6B` base revision locally through the same vLLM version, LFM2 parser, tokenizer, BF16 precision, 32K context window, Compact prompt, tools, and benchmark harness used for Compass.--The base model produced 80 first-turn structured calls, four exact-reference scenario actions, and 34 expected-tool-name outcomes. Compass produced 81, four, and **42** respectively. In this deliberately narrow measurement, the fine-tune does not change the exact-reference count, but it adds **eight** expected Compact routing outcomes.+The base-model row is the comparison we care about most. We served the pinned `LiquidAI/LFM2.5-2.6B` base revision locally through the same vLLM version, LFM2 parser, tokenizer, BF16 precision, 32K context window, Compact prompt, tools, and benchmark harness we used for Compass.-That is a useful, bounded result: Compass is not claiming a broad general-intelligence jump over its base. It is demonstrating that WebBrain-specific training can move the next-action routing policy in the desired direction without making the model larger.+The base model produced 80 first-turn structured calls, four exact-reference scenario actions, and 34 expected-tool-name outcomes. Compass produced 81, four, and 42. In this deliberately narrow measurement the fine-tune leaves the exact-reference count where it was and adds eight expected Compact routing outcomes.-## Why Qwen3.8 is still the default quality choice+That is a bounded result and we would rather report it as one. Compass is not making a broad general-intelligence jump over its base. WebBrain-specific training moved the next-action routing policy in the direction we wanted, without making the model any bigger.-Qwen3.8-27B remains the best model in this comparison if the budget and deployment footprint are acceptable. It has the highest exact-reference score, almost universal first-turn tool-call emission, and fast scenario completion in this run.+## Where the larger models still win-Compass is not positioned as a replacement for that class. The point is different: **a 2.6B specialist can already be useful for a well-scoped WebBrain Compact decision**, rather than forcing every low-risk browser step through a much larger planner.+Qwen3.8-27B is the best model in this comparison if its budget and deployment footprint are acceptable. It has the highest exact-reference score, it emits a first-turn tool call almost every time, and it completed scenarios quickly in this run.-Qwen3.6-35B-A3B remains an important reference point too. Its tie with Compass on expected tool name should not be over-read as equal general intelligence. It is evidence of task specialization, not a claim about broad reasoning, coding, knowledge, multilingual breadth, or long-horizon browser reliability.+Compass is not positioned as a replacement for that class. The claim is smaller: a 2.6B specialist can already handle a well-scoped WebBrain Compact decision, so every low-risk browser step no longer has to go through a much larger planner.-## A note on the public free route--We also exercised the public `liquid/lfm-2.5-2.6b:free` route on OpenRouter as an availability probe. Its scenario service was not stable enough to rank: it returned intermittent provider errors even when requests were serialized. It is therefore excluded from the table above. The local, pinned base comparison is the relevant fine-tune baseline.+Qwen3.6-35B-A3B is an important reference point too, and its tie with Compass on expected tool name should not be over-read as equal general intelligence. It is evidence of task specialization. It says nothing about broad reasoning, coding, knowledge, multilingual breadth, or long-horizon browser reliability.## What happens next-This is the beginning of the evaluation, not the end of it. We are adding more browser surfaces, longer trajectories, tool-result recovery, calibrated escalation behavior, and broader multilingual cases. We also want to measure locally hosted base LFM under the same serving stack before publishing a fine-tune-versus-base headline.+The evaluation is just getting started. We are adding more browser surfaces, longer trajectories, tool-result recovery, calibrated escalation behavior, and broader multilingual cases.-The working roadmap includes Compass **v1.1** and eventually **v2**, with more data, more targeted evaluations, and clearer release artifacts. We hope that work turns this private preview into a useful open-weights option for teams that want WebBrain-native Compact behavior without making a large cloud model the only route for every action.+The working roadmap has Compass v1.1 and eventually v2, with more data, more targeted evaluations, and clearer release artifacts. If that work lands, this private preview becomes an open-weights option for teams that want WebBrain-native Compact behavior without routing every action through a large cloud model.## Reproducibility notes-The comparison used the same Compact Chrome harness and structured tool schemas for every listed model. The runner captures the next response; it does not execute the browser action, so these figures are routing metrics rather than end-to-end success rates. Provider routes, sampling behavior, and model revisions can change, so these results should be read as a dated benchmark snapshot rather than a permanent leaderboard.--More evaluation results will follow as the suite grows.+Every model in the table ran against the same Compact Chrome harness and the same structured tool schemas. The runner captures the next response and does not execute the browser action, so these figures are routing metrics rather than end-to-end success rates. Provider routes, sampling behavior, and model revisions all change over time, so read this as a dated benchmark snapshot rather than a standing leaderboard.Tags: #WebBrain #CompassTiny #BrowserAgent #ToolCalling #EdgeAI #CompactMode #LFM25 #OpenWeightsdiff --git a/web/blog/webbrain-compass-tiny-v1/index.html b/web/blog/webbrain-compass-tiny-v1/index.htmlindex 48d8141c5..e75a8a93a 100644--- a/web/blog/webbrain-compass-tiny-v1/index.html+++ b/web/blog/webbrain-compass-tiny-v1/index.html@@ -4,10 +4,10 @@<meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>Introducing WebBrain Compass Tiny v1 - WebBrain Blog</title>- <meta name="description" content="WebBrain Compass Tiny v1 is a private 2.6B WebBrain Compact-model preview. In our first routing suite it ties Qwen3.6-35B-A3B on tool-name conformity, improves the same local LFM2.5 base model by eight routing outcomes, and keeps the model small enough for edge-oriented deployment.">+ <meta name="description" content="WebBrain Compass Tiny v1 is a private preview of a 2.6B model built for WebBrain's Compact tool loop. In our first routing suite it matches Qwen3.6-35B-A3B on tool-name conformity and beats the local LFM2.5 base model it was trained from by eight routing outcomes, at a size meant for edge deployment."><meta name="keywords" content="WebBrain Compass Tiny, browser agent, tool calling, compact mode, edge AI, LFM2.5, Qwen"><meta property="og:title" content="WebBrain Compass Tiny v1: Compact browser-tool routing at the edge">- <meta property="og:description" content="A first look at WebBrain's 2.6B Compact model preview, its tool-routing benchmark, and the road toward open weights.">+ <meta property="og:description" content="A first look at WebBrain's 2.6B Compact model preview, its tool-routing benchmark, and what an open-weights release would take."><meta property="og:type" content="article"><meta property="og:url" content="https://webbrain.one/blog/webbrain-compass-tiny-v1"><meta property="og:image" content="https://webbrain.one/og-image.png">@@ -18,7 +18,7 @@<meta property="og:image:alt" content="WebBrain — Open-source AI browser agent"><meta name="twitter:card" content="summary_large_image"><meta name="twitter:title" content="WebBrain Compass Tiny v1: an edge-sized Compact model">- <meta name="twitter:description" content="Compass Tiny v1 ties a 35B Qwen model on WebBrain Compact tool-name routing. More tests and v1.1/v2 work are coming.">+ <meta name="twitter:description" content="Compass Tiny v1 matches a 35B Qwen model on WebBrain Compact tool-name routing. More tests and v1.1/v2 work are coming."><meta name="twitter:image" content="https://webbrain.one/twitter-image.png"><meta name="twitter:image:alt" content="WebBrain — Open-source AI browser agent"><link rel="icon" type="image/png" href="/favicon.png">@@ -475,7 +475,7 @@"@context": "https://schema.org","@type": "BlogPosting","headline": "Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain",- "description": "WebBrain Compass Tiny v1 is a private 2.6B WebBrain Compact-model preview. In our first routing suite it ties Qwen3.6-35B-A3B on tool-name conformity, improves the same local LFM2.5 base model by eight routing outcomes, and keeps the model small enough for edge-oriented deployment.",+ "description": "WebBrain Compass Tiny v1 is a private preview of a 2.6B model built for WebBrain's Compact tool loop. In our first routing suite it matches Qwen3.6-35B-A3B on tool-name conformity and beats the local LFM2.5 base model it was trained from by eight routing outcomes, at a size meant for edge deployment.","author": {"@type": "Person","name": "Emre Sokullu",@@ -525,35 +525,31 @@<article><div class="meta">September 5, 2026 · 6 min read · <a href="/blog">← All posts</a></div><h1>Introducing WebBrain Compass Tiny: edge-sized Compact tool routing for WebBrain</h1>- <p class="lede"><strong>WebBrain Compass Tiny v1 is our first edge-sized model built specifically for the Compact browser-tool loop.</strong> It is not a claim that 2.6B parameters equal a 35B model at general reasoning. It is evidence that a focused, grounded routing policy can make a small model useful inside a real browser agent: in this first suite, Compass ties Qwen3.6-35B-A3B on the expected tool name and improves the matched local LFM2.5-2.6B base model by eight routing outcomes. Qwen3.8-27B remains the quality leader when its cost and footprint fit the deployment.</p>+ <p class="lede"><strong>WebBrain Compass Tiny v1 is our first edge-sized model built for the Compact browser-tool loop.</strong> We are not claiming that 2.6B parameters match a 35B model at general reasoning. What this first suite shows is narrower: a focused, grounded routing policy can make a small model useful inside a real browser agent. Compass matches Qwen3.6-35B-A3B at picking the expected tool name, and it beats the LFM2.5-2.6B base model it was trained from by eight routing outcomes. Qwen3.8-27B is still the quality leader wherever its cost and footprint fit.</p><h2 id="the-short-version">The short version</h2>-<p>Compass Tiny v1 is a <strong>private preview</strong> today. We are preparing an open-weights release so it can be downloaded and run locally, subject to the remaining release, provenance, and packaging checks. Until that work is complete, it is not a public download and this article should not be read as a release announcement.</p>+<p>Compass Tiny v1 is a private preview today. We are preparing an open-weights release so it can be downloaded and run locally, once the remaining release, provenance, and packaging checks are done. Until that work is finished it is not a public download, and this article is not a release announcement.</p><p>The model is tuned for three Compact behaviors in WebBrain:</p><ol><li>Answer or ask a clarifying question when that is the justified next step.</li><li>Emit a grounded Compact tool call when the page context supports it.</li><li>Abstain or escalate when direct execution would be unjustified.</li></ol>-<p>That is deliberately narrower than “be a universally smarter model.” It is the right narrowness for a browser agent that must choose a next action from a constrained toolset.</p>+<p>That is narrower than "be a universally smarter model," and the narrowness is the point. A browser agent has to choose its next action from a constrained toolset.</p><h2 id="what-we-measured">What we measured</h2>-<p>We ran the current Compact Chrome suite: 100 first-action cases and 100 stateful scenarios. Eleven scenarios are intentionally out of scope for Compact and are excluded from the scenario-quality denominator, leaving 89 scored scenarios.</p>-<p>The table uses two different signals:</p>-<ul><li><strong>Exact reference action</strong> means the tool and arguments matched the canonical next action.</li><li><strong>Expected tool name</strong> credits the expected tool even when arguments differ. It is useful for measuring routing, but it is not a completed browser task.</li></ul>-<p>The exact reference is a Sonnet-authored canonical next action. It is a regression-oriented reference, not an oracle for every reasonable live-browser strategy: a careful model may inspect before a direct click, for example. That distinction matters here.</p>+<p>We ran the current Compact Chrome suite: 100 first-action cases and 100 stateful scenarios. Eleven of the scenarios are intentionally out of scope for Compact, so they are excluded from the scenario-quality denominator. That leaves 89 scored scenarios.</p>+<p>The table reports two different signals. Exact reference action means the tool and the arguments matched the canonical next action. Expected tool name credits the model for reaching for the right tool even when the arguments differ, which measures routing rather than a completed browser task.</p>+<p>The canonical next action was authored by Sonnet. It works as a regression reference, not as an oracle for every reasonable live-browser strategy: a careful model might inspect something before clicking it directly. Keep that in mind when reading the exact-reference column.</p><div class="table-wrap"><table><thead><tr><th>Model</th><th>Parameters</th><th>First-turn structured calls</th><th>Exact reference action (of 89)</th><th>Expected tool name (of 89)</th></tr></thead><tbody><tr><td>Qwen3.8-27B</td><td>27B</td><td><strong>99 / 100</strong></td><td><strong>17</strong></td><td>39</td></tr><tr><td>Qwen3.6-35B-A3B</td><td>35B A3B</td><td>89 / 100</td><td>15</td><td><strong>42</strong></td></tr><tr><td><strong>WebBrain Compass Tiny v1</strong></td><td><strong>2.6B</strong></td><td><strong>81 / 100</strong></td><td>4</td><td><strong>42</strong></td></tr><tr><td>LFM2.5-2.6B base (local)</td><td>2.6B</td><td>80 / 100</td><td>4</td><td>34</td></tr><tr><td>Qwen3.5-9B</td><td>9B</td><td>84 / 100</td><td>2</td><td>12</td></tr></tbody></table></div>-<p>Compass therefore does not beat the 35B model on the strict reference. It <strong>matches it on tool-name routing</strong> in this Compact suite. It also exceeds the 9B Qwen result by 30 expected-tool-name outcomes and the matched local LFM base model by eight. Those are promising results for a model intended to live near the edge of the product, where size, VRAM, latency, and controllability matter as much as raw general capability.</p>+<p>So Compass does not beat the 35B model on the strict reference. It matches it on tool-name routing in this Compact suite. It also lands 30 expected-tool-name outcomes above the 9B Qwen and eight above the local LFM base model. For a model meant to run near the edge of the product, where size, VRAM, latency, and controllability count for as much as raw general capability, that is a good starting position.</p><h2 id="the-controlled-basemodel-comparison">The controlled base-model comparison</h2>-<p>The base-model row is the comparison we care about most. We served the pinned <code>LiquidAI/LFM2.5-2.6B</code> base revision locally through the same vLLM version, LFM2 parser, tokenizer, BF16 precision, 32K context window, Compact prompt, tools, and benchmark harness used for Compass.</p>-<p>The base model produced 80 first-turn structured calls, four exact-reference scenario actions, and 34 expected-tool-name outcomes. Compass produced 81, four, and <strong>42</strong> respectively. In this deliberately narrow measurement, the fine-tune does not change the exact-reference count, but it adds <strong>eight</strong> expected Compact routing outcomes.</p>-<p>That is a useful, bounded result: Compass is not claiming a broad general-intelligence jump over its base. It is demonstrating that WebBrain-specific training can move the next-action routing policy in the desired direction without making the model larger.</p>-<h2 id="why-qwen3-8-is-still-the-default-quality-choice">Why Qwen3.8 is still the default quality choice</h2>-<p>Qwen3.8-27B remains the best model in this comparison if the budget and deployment footprint are acceptable. It has the highest exact-reference score, almost universal first-turn tool-call emission, and fast scenario completion in this run.</p>-<p>Compass is not positioned as a replacement for that class. The point is different: <strong>a 2.6B specialist can already be useful for a well-scoped WebBrain Compact decision</strong>, rather than forcing every low-risk browser step through a much larger planner.</p>-<p>Qwen3.6-35B-A3B remains an important reference point too. Its tie with Compass on expected tool name should not be over-read as equal general intelligence. It is evidence of task specialization, not a claim about broad reasoning, coding, knowledge, multilingual breadth, or long-horizon browser reliability.</p>-<h2 id="a-note-on-the-public-free-route">A note on the public free route</h2>-<p>We also exercised the public <code>liquid/lfm-2.5-2.6b:free</code> route on OpenRouter as an availability probe. Its scenario service was not stable enough to rank: it returned intermittent provider errors even when requests were serialized. It is therefore excluded from the table above. The local, pinned base comparison is the relevant fine-tune baseline.</p>+<p>The base-model row is the comparison we care about most. We served the pinned <code>LiquidAI/LFM2.5-2.6B</code> base revision locally through the same vLLM version, LFM2 parser, tokenizer, BF16 precision, 32K context window, Compact prompt, tools, and benchmark harness we used for Compass.</p>+<p>The base model produced 80 first-turn structured calls, four exact-reference scenario actions, and 34 expected-tool-name outcomes. Compass produced 81, four, and 42. In this deliberately narrow measurement the fine-tune leaves the exact-reference count where it was and adds eight expected Compact routing outcomes.</p>+<p>That is a bounded result and we would rather report it as one. Compass is not making a broad general-intelligence jump over its base. WebBrain-specific training moved the next-action routing policy in the direction we wanted, without making the model any bigger.</p>+<h2 id="where-the-larger-models-still-win">Where the larger models still win</h2>+<p>Qwen3.8-27B is the best model in this comparison if its budget and deployment footprint are acceptable. It has the highest exact-reference score, it emits a first-turn tool call almost every time, and it completed scenarios quickly in this run.</p>+<p>Compass is not positioned as a replacement for that class. The claim is smaller: a 2.6B specialist can already handle a well-scoped WebBrain Compact decision, so every low-risk browser step no longer has to go through a much larger planner.</p>+<p>Qwen3.6-35B-A3B is an important reference point too, and its tie with Compass on expected tool name should not be over-read as equal general intelligence. It is evidence of task specialization. It says nothing about broad reasoning, coding, knowledge, multilingual breadth, or long-horizon browser reliability.</p><h2 id="what-happens-next">What happens next</h2>-<p>This is the beginning of the evaluation, not the end of it. We are adding more browser surfaces, longer trajectories, tool-result recovery, calibrated escalation behavior, and broader multilingual cases. We also want to measure locally hosted base LFM under the same serving stack before publishing a fine-tune-versus-base headline.</p>-<p>The working roadmap includes Compass <strong>v1.1</strong> and eventually <strong>v2</strong>, with more data, more targeted evaluations, and clearer release artifacts. We hope that work turns this private preview into a useful open-weights option for teams that want WebBrain-native Compact behavior without making a large cloud model the only route for every action.</p>+<p>The evaluation is just getting started. We are adding more browser surfaces, longer trajectories, tool-result recovery, calibrated escalation behavior, and broader multilingual cases.</p>+<p>The working roadmap has Compass v1.1 and eventually v2, with more data, more targeted evaluations, and clearer release artifacts. If that work lands, this private preview becomes an open-weights option for teams that want WebBrain-native Compact behavior without routing every action through a large cloud model.</p><h2 id="reproducibility-notes">Reproducibility notes</h2>-<p>The comparison used the same Compact Chrome harness and structured tool schemas for every listed model. The runner captures the next response; it does not execute the browser action, so these figures are routing metrics rather than end-to-end success rates. Provider routes, sampling behavior, and model revisions can change, so these results should be read as a dated benchmark snapshot rather than a permanent leaderboard.</p>-<p>More evaluation results will follow as the suite grows.</p>+<p>Every model in the table ran against the same Compact Chrome harness and the same structured tool schemas. The runner captures the next response and does not execute the browser action, so these figures are routing metrics rather than end-to-end success rates. Provider routes, sampling behavior, and model revisions all change over time, so read this as a dated benchmark snapshot rather than a standing leaderboard.</p><p>Tags: #WebBrain #CompassTiny #BrowserAgent #ToolCalling #EdgeAI #CompactMode #LFM25 #OpenWeights</p><div class="author-box">