Skip to content

A team of long-lived bots for DeepSeek Harness, with main bots, bot-to-bot messages and group chats.

Connect Bot browser sessions to isolated views

Fei2411a550ca3f4740
diff --git a/browser-plugins/test/write-safety.test.js b/browser-plugins/test/write-safety.test.js
index 9375f1dd..3b1a0838 100644
--- a/browser-plugins/test/write-safety.test.js
+++ b/browser-plugins/test/write-safety.test.js
@@ -123,7 +123,7 @@ async function waitFor(check) {
}
}
-async function setup(t, actionTimeout = 500) {
+async function setup(t, actionTimeout = 500, extra = {}) {
const fixture = await fixtureServer()
let profileRoot
let session
@@ -142,6 +142,7 @@ async function setup(t, actionTimeout = 500) {
downloadDir: join(profileRoot, 'downloads'),
playwrightUrl,
actionTimeout,
+ ...extra,
})
return { fixture, profileRoot, session }
}
@@ -213,6 +214,26 @@ test('payment buttons are refused and card details are never typed', async (t) =
assert.doesNotMatch(after, /Paid/)
})
+test('Bot handoff can resume, but user takeover blocks browser actions until give-back', async t => {
+ let hold = null
+ const { fixture, session } = await setup(t, 500, {
+ getHold: () => hold,
+ onHold: value => { hold = value },
+ })
+ const snapshot = await session.open(fixture.url)
+ assert.match(session.handoff('login', 'Sign in'), /Waiting for the user/)
+ assert.equal(hold.by, 'bot')
+ await assert.rejects(session.click(findRef(snapshot, 'Read link')), /browser is waiting for the user/)
+ assert.equal(session.resume(), 'The browser is yours again.')
+ assert.equal(hold, null)
+
+ hold = { by: 'user', reason: 'takeover', note: '' }
+ assert.throws(() => session.resume(), /Only the user can give it back/)
+ await assert.rejects(session.open(fixture.url), /The user has taken over this browser/)
+ hold = null
+ assert.match(await session.open(fixture.url), /Write safety/)
+})
+
test('POST submit clicks are gated before the server receives them', async (t) => {
const { fixture, session } = await setup(t)
let allowed = true
diff --git a/docs/plans/2026-10-08-browser-and-secrets.md b/docs/plans/2026-10-08-browser-and-secrets.md
index fdfbd243..5c16685d 100644
--- a/docs/plans/2026-10-08-browser-and-secrets.md
+++ b/docs/plans/2026-10-08-browser-and-secrets.md
@@ -20,9 +20,9 @@ Bot 要替用户在网上做事,就会碰到两类东西:网页,以及登
- 每个 Bot 有自己的浏览器面板,可以开多个标签页。每个标签页是一个 Electron webview,放在这个 Bot 自己的存储分区里(`bot:<id>`)。同一个 Bot 的标签页共用登录状态,不同 Bot 之间的 cookie 和登录状态不互通。
- DSH Desktop 只把分区放在内存里,退出就没了。桌面应用把分区存在应用自己的目录里(不在数据位置里),退出再打开还是登录状态;删掉 Bot 时一起清掉。
- 桌面应用自己提供浏览器桥,接口和 DSH Desktop 的一样。主进程只让它发过租约的 webview 挂上,并固定 webview 的设置:开沙箱、不给 Node、拒绝权限请求和下载、不许打开应用自己的本机端口;页面要开新窗口时改成同一个 Bot 的新标签。
-- 页面由用户操作。Bot 的 `read_browser` 只读当前标签页:标题、地址,以及可见内容转成的 Markdown。链接和控件在文中标成 `[#n]`,并列出目标。长页面分段读,用 `from` 接着读。
+- DSH Desktop 的页面由用户操作。Bot 的 `read_browser` 只读当前标签页:标题、地址,以及可见内容转成的 Markdown。桌面应用中,配置浏览器插件后 Bot 通过 Playwright 操作与用户面板相同的页面;接管和交还由用户控制。
- 能读进 Web Components(open shadow DOM)和同源 iframe。读不了跨域 iframe(只给出它们的名字)、canvas 上画的字和隐藏的内容。
-- `read_browser` **不能点击、输入、滚动或打开页面**。工具说明里要求 Bot 把页面内容当作数据,不当作指令。
+- 在 DSH Desktop,`read_browser` **不能点击、输入、滚动或打开页面**。工具说明里要求 Bot 把页面内容当作数据,不当作指令。
- 页面在应用窗口里,不在插件进程里。所以读取请求要经过窗口:客户端一直挂着一个长轮询(`browser-wait`),报告哪些 Bot 开着页面,并把读取请求带回窗口,窗口用 `browser-result` 回答。请求在窗口回答之前一直排队;轮询中断过的会重新发,窗口按 id 去重。
- 所有 Bot、所有标签页加起来最多同时保留 6 个活着的页面(每个都是一个渲染进程)。超过时释放最久没看的那个,地址和摘要保留,再显示时重新加载。
- 能打开本机和局域网的页面。云主机的元数据服务不能打开,因为它会发放账号凭据。
@@ -57,7 +57,7 @@ Bot 要替用户在网上做事,就会碰到两类东西:网页,以及登
## 不做什么
-- Bot 不操作浏览器:不点击、不输入、不打开页面。
+- DSH Desktop 中 Bot 不操作浏览器:不点击、不输入、不打开页面。桌面应用的 Bot 浏览器行为见[浏览器插件](2026-10-21-browser-plugins.md)。
- 不让 Bot 看到或输入密钥的值,不把值写进聊天记录、记忆、DSH 的 profile 或模型请求。
- 联网搜索不借用别的路由的 key。
@@ -78,3 +78,4 @@ Bot 要替用户在网上做事,就会碰到两类东西:网页,以及登
- 2026-10-23:桌面应用里也有 Bot 浏览器,登录保存在应用自己的目录里,#449。
- 2026-10-23:桌面应用的 Bot 浏览器能导入 cookie(文件和本机浏览器),#458。
- 2026-10-21:操作浏览器改由可选插件来做,见 [浏览器插件](2026-10-21-browser-plugins.md)。不配插件时,这一篇的只读行为不变。
+- 2026-10-24:桌面应用中 Bot 操作浏览器面板的页面;用户接管、停止和交还控制权。用户接管只能由用户结束。
diff --git a/docs/plans/2026-10-21-browser-plugins.md b/docs/plans/2026-10-21-browser-plugins.md
index b8854294..125bb344 100644
--- a/docs/plans/2026-10-21-browser-plugins.md
+++ b/docs/plans/2026-10-21-browser-plugins.md
@@ -1,7 +1,7 @@
# 浏览器插件:全团队一个后端,每个 Bot 一份 profile
- 日期:2026-10-21
-- 相关:改 [浏览器、搜索、密钥和连接器](2026-10-08-browser-and-secrets.md) 里「Bot 不操作浏览器」。维护者确认:那条是等这次做的。
+- 相关:[浏览器、搜索、密钥和连接器](2026-10-08-browser-and-secrets.md)
- 代码:`ds-bot/src/host/browser-plugins.js`、`ds-bot/src/host/browser/`、`browser-plugins/`
## 为什么
@@ -27,7 +27,7 @@ Bot 是长期身份。登录应该跟着 Bot 走,而不是跟着某一段对
- `script`:一次提交一段 JavaScript,帮手已经在作用域里。这是 Hermes `browser_exec` 和 ego heredoc 的形状。
- `task`:一个目标进去,插件自己用 `refs` 的工具跑完,交回答案和步骤。
- 截图作为图片块交给模型。后面的模型按多模态来用,不把 Flash 当成纯文本。
-- 桌面里那个用户自己看的面板还在,它不是这三个插件驱动的 Chromium。两边的 cookie 不自动对齐。
+- 在桌面应用中,Playwright 插件连接到主进程拥有的浏览器页;Bot 和用户看到同一组标签页。DSH Desktop 仍使用独立的 Chromium profile。Bot 的 profile 和下载目录按 Bot 隔离。
## 不做什么
@@ -38,7 +38,7 @@ Bot 是长期身份。登录应该跟着 Bot 走,而不是跟着某一段对
## 已知限制
-- 插件驱动的是 Playwright 的 Chromium profile,不是桌面面板里的 Electron webview。用户暂时不能在面板里看着 Bot 点。
+- DSH Desktop 的浏览器面板仍由用户操作;桌面应用中的面板页由 Bot 和用户共同使用,接管状态由用户控制。
- 单独把 `browser-plugins/*` 装进一个没有 DS Bot 的 DSH profile 时,profile 按 Session id 分开,换段会像官方 browser-use 一样丢掉登录。跟 Bot 走的那份目录只有 DS Bot 挂载时才有。
- `task` 在插件内部再调一次模型,步骤在工具结果里,不在 Bot 自己的逐步工具调用里。
- 这次实测用的是自建多步页面,加上 WebVoyager 里两条 arXiv 任务,不是整套 600 多条。
@@ -48,3 +48,4 @@ Bot 是长期身份。登录应该跟着 Bot 走,而不是跟着某一段对
- 2026-10-21:维护者定下全团队一个后端、每 Bot 一份 profile、cookie 不同步。三个插件和实测在同一分支,默认关闭。
- 2026-10-22:实测模型是 OpenDesign 上的 `deepseek-v4.1-flash`(`https://amr-link.open-design.ai/v1`),密钥用 `OPENDESIGN_API_KEY`。
- 2026-10-23:维护者定下权限规则:默认放行;发帖、上传、下载、删除按预授权或审批;删除强制多核对一轮;付款一律交给用户;预授权可设时限,在设置的浏览器页管理。
+- 2026-10-24:桌面应用中的 Bot 操作浏览器面板里的页面;用户可接管、停止 Bot 或交还控制权。Bot 交接可由 Bot 恢复;用户接管只能由用户交还。
diff --git a/ds-bot/client.js b/ds-bot/client.js
index e226c7c1..00eb3278 100644
--- a/ds-bot/client.js
+++ b/ds-bot/client.js
@@ -298,6 +298,12 @@ var ZH = {
"cookies.txt, JSON, storageState": "cookies.txt、JSON、storageState",
"Copy of its cookies": "复制它的 cookie",
"No browsers found": "没有找到浏览器",
+ "The Bot controls this browser": "Bot 正在控制此浏览器",
+ "The Bot is waiting for you": "Bot 正在等待你",
+ "You control this browser": "你正在控制此浏览器",
+ "The Bot handed this browser to you: {reason}": "Bot 已将此浏览器交给你:{reason}",
+ "Take over": "接管",
+ "Give back": "交还",
"{name} is running": "{name} 正在运行",
"It must close for a moment so its cookies can be copied. It will reopen with your tabs.": "需要先把它关掉一下才能复制 cookie,之后会自动重新打开并恢复你的标签页。",
"Close {name} and import": "关闭 {name} 并导入",
@@ -16304,6 +16310,11 @@ function DetailsDrawer({ sessionId, startRenaming, startTab, hasStrip, roster, a
];
}
+// src/client/browser-overlap.js
+function rectsOverlap(a, b) {
+ return a.width > 0 && a.height > 0 && b.width > 0 && b.height > 0 && a.left < b.right && a.right > b.left && a.top < b.bottom && a.bottom > b.top;
+}
+
// src/client/browser-pane.js
var import_react42 = require("react");
var import_react_dom8 = require("react-dom");
@@ -16627,11 +16638,12 @@ function PanelStrip({ bot, subject, view, ids, active, sides, side, canBrowse, r
(0, import_react42.createElement)("button", { type: "button", className: "bt-icon-btn bt-ptab-close", "aria-label": t("Close panel"), title: t("Close panel"), onClick: () => actions.closePanel() }, (0, import_react42.createElement)(CloseIcon2))
);
}
-function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages }) {
+function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages, hold }) {
const page = useBrowserPages((pages) => pages[tabId ?? ""]) ?? EMPTY_PAGE;
const frame = (0, import_react42.useRef)(null);
const input = (0, import_react42.useRef)(null);
const [draft, setDraft] = (0, import_react42.useState)(null);
+ const [covered, setCovered] = (0, import_react42.useState)(false);
const imports = actions.browserImport;
const [flow, setFlow] = (0, import_react42.useState)(null);
const flowRef = (0, import_react42.useRef)(null);
@@ -16641,7 +16653,22 @@ function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages })
flowRef.current = typeof next === "function" ? next(flowRef.current) : next;
setFlow(flowRef.current);
};
- const showGuest = landed && !leaving && Boolean(page.url) && !page.error && flow === null;
+ const showGuest = landed && !leaving && !covered && Boolean(page.url) && !page.error && flow === null;
+ (0, import_react42.useEffect)(() => {
+ const update = () => {
+ const panel = frame.current?.getBoundingClientRect();
+ const overlaps2 = panel && [...document.querySelectorAll('[role="dialog"],[role="menu"],[role="listbox"]')].some((element) => rectsOverlap(panel, element.getBoundingClientRect()));
+ setCovered(Boolean(overlaps2));
+ };
+ const observer = new MutationObserver(update);
+ observer.observe(document.body, { subtree: true, childList: true, attributes: true, attributeFilter: ["role", "data-leaving"] });
+ window.addEventListener("resize", update);
+ update();
+ return () => {
+ observer.disconnect();
+ window.removeEventListener("resize", update);
+ };
+ }, []);
(0, import_react42.useEffect)(() => {
setFlowTracked(null);
return () => {
@@ -16731,6 +16758,17 @@ function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages })
return (0, import_react42.createElement)(
"div",
{ className: "bt-panel-view bt-panel-browser", "aria-label": t("{name}'s browser", { name: bot.name }) },
+ actions.browserTakeover ? (0, import_react42.createElement)(
+ "div",
+ { className: "bt-browser-takeover", "data-held": hold?.by === "user" || void 0 },
+ (0, import_react42.createElement)("span", null, hold?.by === "user" ? t("You control this browser") : hold?.by === "bot" ? t("The Bot handed this browser to you: {reason}", { reason: hold.reason }) : t("The Bot controls this browser")),
+ hold?.by ? (0, import_react42.createElement)("button", { type: "button", onClick: () => void actions.browserGiveBack(bot.id) }, t("Give back")) : (0, import_react42.createElement)(
+ "span",
+ { className: "bt-browser-takeover-actions" },
+ (0, import_react42.createElement)("button", { type: "button", onClick: () => void actions.browserTakeover(bot.id) }, t("Take over")),
+ (0, import_react42.createElement)("button", { type: "button", onClick: () => void actions.browserStop(bot.id) }, t("Stop"))
+ )
+ ) : null,
(0, import_react42.createElement)(
"div",
{ className: "bt-browser-bar" },
@@ -16968,7 +17006,7 @@ function ImportCard({ flow, page, busy, onApply, onCancel, onCloseAndImport, onD
)
));
}
-function BotPanel({ pane, leaving, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages }) {
+function BotPanel({ pane, leaving, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages, useBrowserHolds }) {
const bot = roster.byId[pane.id];
const canBrowse = Boolean(bot && roster.browser === true && actions.browserShow);
const sideChats = bot ? sideChatsOf(roster, bot.id) : [];
@@ -16977,6 +17015,7 @@ function BotPanel({ pane, leaving, roster, actions, useSessions, useSessionStatu
const tabState = useBrowserTabs((map) => bot ? map[bot.id] ?? EMPTY_TABS2 : EMPTY_TABS2);
const active = tabState.active;
const page = useBrowserPages((pages) => (active ? pages[active] : null) ?? EMPTY_PAGE);
+ const hold = useBrowserHolds?.((map) => bot ? map[bot.id] ?? null : null) ?? null;
const root = (0, import_react42.useRef)(null);
const fly = (0, import_react42.useRef)(null);
const [landed, setLanded] = (0, import_react42.useState)(false);
@@ -17199,7 +17238,7 @@ function BotPanel({ pane, leaving, roster, actions, useSessions, useSessionStatu
// pages underneath (the chat, the guest) see no pointer.
dragging ? (0, import_react_dom8.createPortal)((0, import_react42.createElement)("div", { className: "bt-drag-shield", "aria-hidden": true }), document.body) : null,
hasStrip ? (0, import_react42.createElement)(PanelStrip, { bot, subject: pane.id, view, ids: tabState.ids, active, sides: sideChats, side: pane.side, canBrowse, reading, onView: switchView, actions: panelActions, useBrowserPages }) : null,
- view === "browser" && bot ? (0, import_react42.createElement)(BrowserView, { key: `browser:${active}`, bot, tabId: active, leaving, landed: landed && !settling, actions, useBrowserPages }) : view === "side" && sideChat ? (0, import_react42.createElement)("div", { key: `side:${sideChat.id}`, className: "bt-panel-view" }, (0, import_react42.createElement)(SideChatView, { subject: pane.id, chat: sideChat, actions: panelActions })) : (0, import_react42.createElement)(
+ view === "browser" && bot ? (0, import_react42.createElement)(BrowserView, { key: `browser:${active}`, bot, tabId: active, leaving, landed: landed && !settling, actions, useBrowserPages, hold }) : view === "side" && sideChat ? (0, import_react42.createElement)("div", { key: `side:${sideChat.id}`, className: "bt-panel-view" }, (0, import_react42.createElement)(SideChatView, { subject: pane.id, chat: sideChat, actions: panelActions })) : (0, import_react42.createElement)(
"div",
{ key: `details:${pane.id}:${pane.renaming ? "rename" : ""}:${pane.tab ?? ""}`, className: "bt-panel-view" },
(0, import_react42.createElement)(DetailsDrawer, {
@@ -17256,6 +17295,10 @@ var BROWSER_CSS = `
.bt-panel[data-settling]>.bt-ptabs,.bt-panel[data-settling]>.bt-panel-view{flex-shrink:0;align-self:flex-end;box-sizing:border-box;width:calc(var(--bt-panel-w,320px) - .5px)}
.bt-panel-browser{position:relative}
.bt-browser-bar{position:relative;display:flex;align-items:center;gap:2px;height:52px;padding:0 10px 0 12px;flex:none;border-bottom:.5px solid var(--bt-line)}
+.bt-browser-takeover{display:flex;align-items:center;justify-content:space-between;gap:10px;padding:8px 12px;background:var(--bt-hover);border-bottom:.5px solid var(--bt-line);font-size:12px;color:var(--bt-ink-2)}
+.bt-browser-takeover[data-held]{background:var(--bt-active)}
+.bt-browser-takeover-actions{display:flex;gap:6px}
+.bt-browser-takeover button{border:1px solid var(--bt-line-2);border-radius:8px;padding:5px 9px;background:var(--bt-main);color:var(--bt-ink);font:inherit;cursor:pointer}
.bt-browser-bar .bt-icon-btn:disabled{opacity:.35;cursor:default;background:none}
.bt-browser-address{flex:1;min-width:0;display:flex;align-items:center;gap:6px;height:32px;margin:0 4px;padding:0 12px;border-radius:999px;background:var(--bt-hover);border:.5px solid transparent;transition:background-color .12s ease,border-color .12s ease,box-shadow .12s ease}
.bt-browser-address:hover{background:var(--bt-active)}
@@ -17343,7 +17386,7 @@ var import_react43 = require("react");
var import_react_dom9 = require("react-dom");
var DIALOG_EXIT_MS = 160;
var MENU_EXIT_MS3 = 120;
-function Overlays({ useRoster, useUi, useSessions, useSessionStatus, useActivity, useWork, useRegistry, useBrowserPages, useBrowserTabs, useUpdate, actions }) {
+function Overlays({ useRoster, useUi, useSessions, useSessionStatus, useActivity, useWork, useRegistry, useBrowserPages, useBrowserTabs, useBrowserHolds, useUpdate, actions }) {
const roster = useRoster((value) => value);
const ui2 = useUi((value) => value);
useRegistry((value) => value);
@@ -17378,7 +17421,7 @@ function Overlays({ useRoster, useUi, useSessions, useSessionStatus, useActivity
return (0, import_react43.createElement)(
import_react43.Fragment,
null,
- pane ? (0, import_react_dom9.createPortal)((0, import_react43.createElement)(BotPanel, { pane, leaving: open === null, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages }), document.body) : null,
+ pane ? (0, import_react_dom9.createPortal)((0, import_react43.createElement)(BotPanel, { pane, leaving: open === null, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages, useBrowserHolds }), document.body) : null,
ui2.newChat ? (0, import_react_dom9.createPortal)((0, import_react43.createElement)(NewChat, { key: ui2.newChat, mode: ui2.newChat, roster, actions }), document.body) : null,
ui2.palette ? (0, import_react_dom9.createPortal)((0, import_react43.createElement)(CommandPalette, { key: ui2.palette.token, start: ui2.palette.query, token: ui2.palette.token, roster, actions, useSessions }), document.body) : null,
ui2.exchange ? (0, import_react_dom9.createPortal)((0, import_react43.createElement)(ExchangeDialog, { pair: ui2.exchange, roster, actions }), document.body) : null,
@@ -17791,6 +17834,73 @@ var PREVIEW_SCRIPT = `(() => {
return text.replace(/\\s+/g, ' ').trim().slice(0, 200)
})()`;
var EMPTY = { url: "", title: "", favicon: null, loading: false, canGoBack: false, canGoForward: false, error: null, live: false, excerpt: "" };
+function createAppViewProxy(bridge, botId, tabId) {
+ const listeners2 = /* @__PURE__ */ new Map();
+ const state = { url: "", title: "", loading: false, canGoBack: false, canGoForward: false };
+ const fire = (name, event = {}) => {
+ for (const listener of listeners2.get(name) ?? []) listener({ type: name, ...event });
+ };
+ return {
+ isAppViewProxy: true,
+ style: {},
+ dataset: {},
+ addEventListener(name, listener, options = {}) {
+ listeners2.set(name, [...listeners2.get(name) ?? [], listener]);
+ options.signal?.addEventListener("abort", () => this.removeEventListener(name, listener), { once: true });
+ },
+ removeEventListener(name, listener) {
+ listeners2.set(name, (listeners2.get(name) ?? []).filter((item) => item !== listener));
+ },
+ setAttribute() {
+ },
+ remove() {
+ void bridge.views.place(tabId, null, false);
+ },
+ getURL() {
+ return state.url;
+ },
+ getTitle() {
+ return state.title;
+ },
+ isLoading() {
+ return state.loading;
+ },
+ canGoBack() {
+ return state.canGoBack;
+ },
+ canGoForward() {
+ return state.canGoForward;
+ },
+ loadURL(url) {
+ return bridge.views.navigate(botId, tabId, url);
+ },
+ executeJavaScript(script) {
+ return bridge.views.read(botId, tabId, script);
+ },
+ goBack() {
+ return bridge.views.command(botId, tabId, "back");
+ },
+ goForward() {
+ return bridge.views.command(botId, tabId, "forward");
+ },
+ reload() {
+ return bridge.views.command(botId, tabId, "reload");
+ },
+ stop() {
+ return bridge.views.command(botId, tabId, "stop");
+ },
+ focus() {
+ return bridge.views.focus(botId, tabId);
+ },
+ update(next) {
+ Object.assign(state, next);
+ fire("did-navigate");
+ fire("did-stop-loading");
+ fire("page-title-updated");
+ if (!state.loading) fire("dom-ready");
+ }
+ };
+}
var METADATA_HOST = /^(169\.254\.\d+\.\d+|\[fd00:ec2::254\]|metadata\.google\.internal)$/i;
var BLOCKED = "Cloud metadata addresses do not open here.";
var blockedAddress = (url) => URL.canParse(url) && METADATA_HOST.test(new URL(url).hostname);
@@ -17805,11 +17915,12 @@ function browserAddress(input) {
if (looksLikeHost && URL.canParse(`http://${text}`)) return new URL(`${local ? "http" : "https"}://${text}`).href;
return `${SEARCH_URL}${encodeURIComponent(text)}`;
}
-function createBotBrowsers({ bridge, createSource: createSource2, storage = globalThis.localStorage, document: doc = globalThis.document }) {
+function createBotBrowsers({ bridge, createSource: createSource2, storage = globalThis.localStorage, document: doc = globalThis.document, guestFactory }) {
const views = /* @__PURE__ */ new Map();
const owner = /* @__PURE__ */ new Map();
const tabs = createSource2({});
const pages = createSource2({});
+ const holds = createSource2({});
const store = sharedMap(STORE_KEY, storage);
let saved = store.read();
const seen = /* @__PURE__ */ new Set();
@@ -17820,6 +17931,57 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
let resizeLater = 0;
let disposed = false;
let serial = 0;
+ const makeGuest = guestFactory ?? (bridge.views ? ({ botId, tabId }) => ({ element: createAppViewProxy(bridge, botId, tabId), appView: true }) : ({ document: document2 }) => ({ element: document2.createElement("webview"), appView: false }));
+ const appViewEvents = bridge.views?.subscribe?.((change) => {
+ const { botId, tabId } = change ?? {};
+ if (!botId || !tabId) return;
+ if (change.type === "closed") {
+ const entry2 = views.get(tabId);
+ entry2?.stop();
+ views.delete(tabId);
+ owner.delete(tabId);
+ tabs.set((snapshot) => {
+ const current = snapshot[botId];
+ if (!current) return snapshot;
+ const ids = current.ids.filter((id) => id !== tabId);
+ if (ids.length === 0) {
+ const next = { ...snapshot };
+ delete next[botId];
+ return next;
+ }
+ return { ...snapshot, [botId]: { ids, active: current.active === tabId ? ids.at(-1) : current.active } };
+ });
+ pages.set((snapshot) => {
+ const { [tabId]: _, ...rest } = snapshot;
+ return rest;
+ });
+ persist(botId);
+ return;
+ }
+ if (!owner.has(tabId)) owner.set(tabId, botId);
+ if (change.type === "created" && change.createdByBot) {
+ ensureTabs(botId);
+ tabs.set((snapshot) => {
+ const own = snapshot[botId] ?? { ids: [], active: null };
+ if (own.ids.includes(tabId)) return snapshot;
+ return { ...snapshot, [botId]: { ids: [...own.ids, tabId], active: tabId } };
+ });
+ publish2(tabId, { url: change.url ?? "", title: change.title ?? "", live: true });
+ persist(botId);
+ void open(tabId);
+ }
+ const entry = views.get(tabId);
+ if (entry?.appView) {
+ entry.botAttached = change.botAttached === true;
+ entry.ready = change.loading !== true;
+ entry.element.update(change);
+ if (entry.ready) {
+ observe(entry);
+ void preview(tabId);
+ }
+ }
+ publish2(tabId, change);
+ });
const savedTab = (botId, tabId) => saved[botId]?.tabs?.find((entry) => entry.id === tabId);
const publish2 = (tabId, patch) => pages.set((state) => {
const botId = owner.get(tabId);
@@ -17855,6 +18017,10 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
};
const place = (entry) => {
const box = rect ?? { left: 0, top: 0, width: 800, height: 600 };
+ if (entry.appView) {
+ void bridge.views.place(entry.tabId, box, shown === entry.tabId && visible && Boolean(rect));
+ return;
+ }
Object.assign(entry.element.style, {
left: `${box.left}px`,
top: `${box.top}px`,
@@ -17896,26 +18062,35 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
};
async function create(tabId) {
const botId = owner.get(tabId);
- const reservation = await bridge.acquire(`bot:${botId}`);
+ const guest = await makeGuest({ bridge, document: doc, botId, tabId });
+ const pending = pages.getSnapshot()[tabId]?.url || savedTab(botId, tabId)?.url;
+ const reservation = guest.appView ? void 0 : await bridge.acquire(`bot:${botId}`);
if (disposed) {
- await bridge.release(reservation.lease).catch(() => {
+ if (reservation) await bridge.release(reservation.lease).catch(() => {
+ });
+ else await bridge.views.close(botId, tabId).catch(() => {
});
return void 0;
}
- const element = doc.createElement("webview");
+ if (guest.appView) await bridge.views.create(botId, tabId, pending ?? "");
+ const element = guest.element;
element.dataset.sidebarBrowserFrame = "webview";
Object.assign(element.style, { position: "fixed", display: "flex", border: "0", pointerEvents: "auto", background: "#fff" });
element.dataset.btBrowser = tabId;
- element.setAttribute("name", reservation.lease);
- element.setAttribute("partition", reservation.partition);
- element.setAttribute("allowpopups", "");
- element.setAttribute("src", `about:blank#${reservation.lease}`);
- const entry = { botId, tabId, element, lease: reservation.lease, ready: false, fresh: true, pending: pages.getSnapshot()[tabId]?.url || savedTab(botId, tabId)?.url, usedAt: Date.now() };
+ if (!guest.appView) {
+ element.setAttribute("name", reservation.lease);
+ element.setAttribute("partition", reservation.partition);
+ element.setAttribute("allowpopups", "");
+ element.setAttribute("src", `about:blank#${reservation.lease}`);
+ }
+ const entry = { botId, tabId, element, lease: reservation?.lease, appView: guest.appView, botAttached: false, ready: false, fresh: !guest.appView, pending: guest.appView ? void 0 : pending, usedAt: Date.now() };
const lifetime = new AbortController();
entry.stop = () => lifetime.abort();
const { signal } = lifetime;
- const unsubscribe = bridge.onOpenRequested(reservation.lease, (url) => void openTab(botId, url));
- signal.addEventListener("abort", unsubscribe, { once: true });
+ if (!guest.appView) {
+ const unsubscribe = bridge.onOpenRequested(reservation.lease, (url) => void openTab(botId, url));
+ signal.addEventListener("abort", unsubscribe, { once: true });
+ }
element.addEventListener("dom-ready", () => {
entry.ready = true;
observe(entry);
@@ -17954,9 +18129,16 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
publish2(tabId, { loading: false, error: "The page crashed." });
}, { signal });
views.set(tabId, entry);
- ensureLayer().append(element);
+ if (!guest.appView) ensureLayer().append(element);
place(entry);
publish2(tabId, { live: true, loading: Boolean(entry.pending) });
+ if (guest.appView) {
+ const state = (await bridge.views.sync(botId)).find((view) => view.tabId === tabId);
+ if (state) {
+ entry.botAttached = state.botAttached === true;
+ element.update(state);
+ }
+ }
return entry;
}
const opening = /* @__PURE__ */ new Map();
@@ -17976,12 +18158,16 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
return job;
}
function trim() {
- const idle = [...views.values()].filter((entry) => entry.tabId !== shown).sort((a, b) => a.usedAt - b.usedAt);
- while (views.size > MAX_LIVE && idle.length > 0) void release(idle.shift().tabId);
+ const idle = [...views.values()].filter((entry) => !entry.appView && !entry.botAttached && entry.tabId !== shown).sort((a, b) => a.usedAt - b.usedAt);
+ while ([...views.values()].filter((entry) => !entry.appView).length > MAX_LIVE && idle.length > 0) void release(idle.shift().tabId);
}
async function release(tabId) {
const entry = views.get(tabId);
if (entry === void 0) return;
+ if (entry.appView) {
+ await bridge.views.place(tabId, rect ?? { left: 0, top: 0, width: 800, height: 600 }, false);
+ return;
+ }
views.delete(tabId);
entry.stop();
entry.element.remove();
@@ -18007,7 +18193,11 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
await opening.get(tabId)?.catch(() => {
});
if (shown === tabId) await show2(null);
- await release(tabId);
+ if (views.get(tabId)?.appView) {
+ await bridge.views.close(botId, tabId);
+ views.get(tabId)?.stop();
+ views.delete(tabId);
+ } else await release(tabId);
tabs.set((snapshot) => {
const current = snapshot[botId];
if (current === void 0 || !current.ids.includes(tabId)) return snapshot;
@@ -18077,7 +18267,11 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
await opening.get(tabId)?.catch(() => {
});
if (shown === tabId) await show2(null);
- await release(tabId);
+ if (views.get(tabId)?.appView) {
+ await bridge.views.close(botId, tabId);
+ views.get(tabId)?.stop();
+ views.delete(tabId);
+ } else await release(tabId);
owner.delete(tabId);
pages.set((snapshot) => {
if (snapshot[tabId] === void 0) return snapshot;
@@ -18097,6 +18291,10 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
return {
tabs,
pages,
+ holds,
+ setHolds(value) {
+ holds.set(value && typeof value === "object" ? value : {});
+ },
/** Restore a Bot's saved strip (or an empty one) so the panel can render it. */
ensureTabs,
openTab,
@@ -18160,7 +18358,11 @@ function createBotBrowsers({ bridge, createSource: createSource2, storage = glob
dispose() {
disposed = true;
clearTimeout(resizeLater);
- for (const entry of views.values()) void release(entry.tabId);
+ for (const entry of views.values()) {
+ if (entry.appView) void bridge.views.place(entry.tabId, rect ?? { left: 0, top: 0, width: 800, height: 600 }, false);
+ else void release(entry.tabId);
+ }
+ appViewEvents?.();
layer?.remove();
}
};
@@ -18195,8 +18397,10 @@ function serveBrowserReads({ browsers, post, clientId = globalThis.crypto.random
const open = browsers.openPages();
reported = JSON.stringify(open);
try {
- const requests = await post("browser-wait", { clientId, open }, poll.signal);
+ const response = await post("browser-wait", { clientId, open }, poll.signal);
+ browsers.setHolds?.(response?.holds);
failures = 0;
+ const requests = Array.isArray(response) ? response : response?.requests;
for (const request of Array.isArray(requests) ? requests : []) void answer(request);
} catch {
if (stopped) return;
@@ -18960,6 +19164,9 @@ function apply(ctx) {
browserNavigate: (tabId, input) => browsers.navigate(tabId, input),
browserCommand: (tabId, name) => browsers.command(tabId, name),
browserFocus: (tabId) => browsers.focus(tabId),
+ browserTakeover: bridge.views ? (botId) => call("browser-takeover", { botId }) : void 0,
+ browserStop: bridge.views ? (botId) => call("browser-stop", { botId }) : void 0,
+ browserGiveBack: bridge.views ? (botId) => call("browser-give-back", { botId }) : void 0,
// Cookie imports, only where the bridge offers them (the desktop app): a file or an
// installed browser's profile is previewed, then applied into the Bot's storage key.
browserImport: typeof bridge.importSources === "function" ? {
@@ -19015,7 +19222,8 @@ function apply(ctx) {
};
const browserPages = browsers?.pages ?? createSource({});
const browserTabs = browsers?.tabs ?? createSource({});
- const face = () => ({ actions, hooks: { roster, ui: ui2, activity, work, memoryNews, registry, turnClock, live, cards, heldCards, browserPages, browserTabs, surface, update } });
+ const browserHolds = browsers?.holds ?? createSource({});
+ const face = () => ({ actions, hooks: { roster, ui: ui2, activity, work, memoryNews, registry, turnClock, live, cards, heldCards, browserPages, browserTabs, browserHolds, surface, update } });
ctx.effect(() => noticeApprovals({
work,
roster,
diff --git a/ds-bot/src/client/bot-browsers.js b/ds-bot/src/client/bot-browsers.js
index 28117d07..1528d883 100644
--- a/ds-bot/src/client/bot-browsers.js
+++ b/ds-bot/src/client/bot-browsers.js
@@ -33,6 +33,42 @@ export const PREVIEW_SCRIPT = `(() => {
})()`
const EMPTY = { url: '', title: '', favicon: null, loading: false, canGoBack: false, canGoForward: false, error: null, live: false, excerpt: '' }
+export function createAppViewProxy(bridge, botId, tabId) {
+ const listeners = new Map()
+ const state = { url: '', title: '', loading: false, canGoBack: false, canGoForward: false }
+ const fire = (name, event = {}) => {
+ for (const listener of listeners.get(name) ?? []) listener({ type: name, ...event })
+ }
+ return {
+ isAppViewProxy: true, style: {}, dataset: {},
+ addEventListener(name, listener, options = {}) {
+ listeners.set(name, [...(listeners.get(name) ?? []), listener])
+ options.signal?.addEventListener('abort', () => this.removeEventListener(name, listener), { once: true })
+ },
+ removeEventListener(name, listener) { listeners.set(name, (listeners.get(name) ?? []).filter(item => item !== listener)) },
+ setAttribute() {},
+ remove() { void bridge.views.place(tabId, null, false) },
+ getURL() { return state.url },
+ getTitle() { return state.title },
+ isLoading() { return state.loading },
+ canGoBack() { return state.canGoBack },
+ canGoForward() { return state.canGoForward },
+ loadURL(url) { return bridge.views.navigate(botId, tabId, url) },
+ executeJavaScript(script) { return bridge.views.read(botId, tabId, script) },
+ goBack() { return bridge.views.command(botId, tabId, 'back') },
+ goForward() { return bridge.views.command(botId, tabId, 'forward') },
+ reload() { return bridge.views.command(botId, tabId, 'reload') },
+ stop() { return bridge.views.command(botId, tabId, 'stop') },
+ focus() { return bridge.views.focus(botId, tabId) },
+ update(next) {
+ Object.assign(state, next)
+ fire('did-navigate')
+ fire('did-stop-loading')
+ fire('page-title-updated')
+ if (!state.loading) fire('dom-ready')
+ },
+ }
+}
// Local and LAN pages open; cloud metadata services do not, since on a cloud machine they
// hand out account credentials. The Desktop shell already refuses its own host port.
const METADATA_HOST = /^(169\.254\.\d+\.\d+|\[fd00:ec2::254\]|metadata\.google\.internal)$/i
@@ -59,7 +95,7 @@ export function browserAddress(input) {
/**
* @param {{ bridge: any, createSource: (initial:any) => any, storage?: Storage, document?: Document }} options
*/
-export function createBotBrowsers({ bridge, createSource, storage = globalThis.localStorage, document: doc = globalThis.document }) {
+export function createBotBrowsers({ bridge, createSource, storage = globalThis.localStorage, document: doc = globalThis.document, guestFactory }) {
/** @type {Map<string, any>} tabId -> guest entry */
const views = new Map()
/** @type {Map<string, string>} tabId -> owning Bot id */
@@ -68,6 +104,7 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
const tabs = createSource({})
// Per tab: { url, title, favicon, loading, canGoBack, canGoForward, error, live, readAt?, excerpt }
const pages = createSource({})
+ const holds = createSource({})
// Per Bot, what survives a reload: { tabs: [{id,url,title}], active }. Another window
// saves its own Bots' tabs into the same object.
const store = sharedMap(STORE_KEY, storage)
@@ -83,6 +120,49 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
let resizeLater = 0
let disposed = false
let serial = 0
+ const makeGuest = guestFactory ?? (bridge.views
+ ? ({ botId, tabId }) => ({ element: createAppViewProxy(bridge, botId, tabId), appView: true })
+ : ({ document }) => ({ element: document.createElement('webview'), appView: false }))
+ const appViewEvents = bridge.views?.subscribe?.((change) => {
+ const { botId, tabId } = change ?? {}
+ if (!botId || !tabId) return
+ if (change.type === 'closed') {
+ const entry = views.get(tabId)
+ entry?.stop()
+ views.delete(tabId)
+ owner.delete(tabId)
+ tabs.set(snapshot => {
+ const current = snapshot[botId]
+ if (!current) return snapshot
+ const ids = current.ids.filter(id => id !== tabId)
+ if (ids.length === 0) { const next = { ...snapshot }; delete next[botId]; return next }
+ return { ...snapshot, [botId]: { ids, active: current.active === tabId ? ids.at(-1) : current.active } }
+ })
+ pages.set(snapshot => { const { [tabId]: _, ...rest } = snapshot; return rest })
+ persist(botId)
+ return
+ }
+ if (!owner.has(tabId)) owner.set(tabId, botId)
+ if (change.type === 'created' && change.createdByBot) {
+ ensureTabs(botId)
+ tabs.set(snapshot => {
+ const own = snapshot[botId] ?? { ids: [], active: null }
+ if (own.ids.includes(tabId)) return snapshot
+ return { ...snapshot, [botId]: { ids: [...own.ids, tabId], active: tabId } }
+ })
+ publish(tabId, { url: change.url ?? '', title: change.title ?? '', live: true })
+ persist(botId)
+ void open(tabId)
+ }
+ const entry = views.get(tabId)
+ if (entry?.appView) {
+ entry.botAttached = change.botAttached === true
+ entry.ready = change.loading !== true
+ entry.element.update(change)
+ if (entry.ready) { observe(entry); void preview(tabId) }
+ }
+ publish(tabId, change)
+ })
const savedTab = (botId, tabId) => saved[botId]?.tabs?.find(entry => entry.id === tabId)
const publish = (tabId, patch) => pages.set((state) => {
@@ -124,6 +204,10 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
const place = (entry) => {
const box = rect ?? { left: 0, top: 0, width: 800, height: 600 }
+ if (entry.appView) {
+ void bridge.views.place(entry.tabId, box, shown === entry.tabId && visible && Boolean(rect))
+ return
+ }
Object.assign(entry.element.style, {
left: `${box.left}px`, top: `${box.top}px`, width: `${box.width}px`, height: `${box.height}px`,
visibility: shown === entry.tabId && visible && rect ? 'visible' : 'hidden',
@@ -162,25 +246,36 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
async function create(tabId) {
const botId = owner.get(tabId)
- const reservation = await bridge.acquire(`bot:${botId}`)
- if (disposed) { await bridge.release(reservation.lease).catch(() => {}); return undefined }
- const element = doc.createElement('webview')
+ const guest = await makeGuest({ bridge, document: doc, botId, tabId })
+ const pending = pages.getSnapshot()[tabId]?.url || savedTab(botId, tabId)?.url
+ const reservation = guest.appView ? undefined : await bridge.acquire(`bot:${botId}`)
+ if (disposed) {
+ if (reservation) await bridge.release(reservation.lease).catch(() => {})
+ else await bridge.views.close(botId, tabId).catch(() => {})
+ return undefined
+ }
+ if (guest.appView) await bridge.views.create(botId, tabId, pending ?? '')
+ const element = guest.element
// The shell forwards app shortcuts from a focused webview that carries these
// attributes, as it does for its own Sidebar browser.
element.dataset.sidebarBrowserFrame = 'webview'
Object.assign(element.style, { position: 'fixed', display: 'flex', border: '0', pointerEvents: 'auto', background: '#fff' })
element.dataset.btBrowser = tabId
- element.setAttribute('name', reservation.lease)
- element.setAttribute('partition', reservation.partition)
- element.setAttribute('allowpopups', '')
- element.setAttribute('src', `about:blank#${reservation.lease}`)
- const entry = { botId, tabId, element, lease: reservation.lease, ready: false, fresh: true, pending: pages.getSnapshot()[tabId]?.url || savedTab(botId, tabId)?.url, usedAt: Date.now() }
+ if (!guest.appView) {
+ element.setAttribute('name', reservation.lease)
+ element.setAttribute('partition', reservation.partition)
+ element.setAttribute('allowpopups', '')
+ element.setAttribute('src', `about:blank#${reservation.lease}`)
+ }
+ const entry = { botId, tabId, element, lease: reservation?.lease, appView: guest.appView, botAttached: false, ready: false, fresh: !guest.appView, pending: guest.appView ? undefined : pending, usedAt: Date.now() }
const lifetime = new AbortController()
entry.stop = () => lifetime.abort()
const { signal } = lifetime
// A page that asks for a new window gets a new tab in the same Bot, activated.
- const unsubscribe = bridge.onOpenRequested(reservation.lease, url => void openTab(botId, url))
- signal.addEventListener('abort', unsubscribe, { once: true })
+ if (!guest.appView) {
+ const unsubscribe = bridge.onOpenRequested(reservation.lease, url => void openTab(botId, url))
+ signal.addEventListener('abort', unsubscribe, { once: true })
+ }
element.addEventListener('dom-ready', () => {
entry.ready = true
observe(entry)
@@ -223,9 +318,16 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
publish(tabId, { loading: false, error: 'The page crashed.' })
}, { signal })
views.set(tabId, entry)
- ensureLayer().append(element)
+ if (!guest.appView) ensureLayer().append(element)
place(entry)
publish(tabId, { live: true, loading: Boolean(entry.pending) })
+ if (guest.appView) {
+ const state = (await bridge.views.sync(botId)).find(view => view.tabId === tabId)
+ if (state) {
+ entry.botAttached = state.botAttached === true
+ element.update(state)
+ }
+ }
return entry
}
@@ -254,13 +356,17 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
// Release the least recently shown guests beyond the limit, never the one on screen.
function trim() {
- const idle = [...views.values()].filter(entry => entry.tabId !== shown).sort((a, b) => a.usedAt - b.usedAt)
- while (views.size > MAX_LIVE && idle.length > 0) void release(idle.shift().tabId)
+ const idle = [...views.values()].filter(entry => !entry.appView && !entry.botAttached && entry.tabId !== shown).sort((a, b) => a.usedAt - b.usedAt)
+ while ([...views.values()].filter(entry => !entry.appView).length > MAX_LIVE && idle.length > 0) void release(idle.shift().tabId)
}
async function release(tabId) {
const entry = views.get(tabId)
if (entry === undefined) return
+ if (entry.appView) {
+ await bridge.views.place(tabId, rect ?? { left: 0, top: 0, width: 800, height: 600 }, false)
+ return
+ }
views.delete(tabId)
entry.stop()
entry.element.remove()
@@ -287,7 +393,11 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
if (state === undefined || !state.ids.includes(tabId)) return
await opening.get(tabId)?.catch(() => {})
if (shown === tabId) await show(null)
- await release(tabId)
+ if (views.get(tabId)?.appView) {
+ await bridge.views.close(botId, tabId)
+ views.get(tabId)?.stop()
+ views.delete(tabId)
+ } else await release(tabId)
// The waits above can overlap a new tab or an active-tab switch; drop the
// target from the latest state, not the snapshot taken before them.
tabs.set((snapshot) => {
@@ -369,7 +479,11 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
for (const tabId of tabs.getSnapshot()[botId]?.ids ?? []) {
await opening.get(tabId)?.catch(() => {})
if (shown === tabId) await show(null)
- await release(tabId)
+ if (views.get(tabId)?.appView) {
+ await bridge.views.close(botId, tabId)
+ views.get(tabId)?.stop()
+ views.delete(tabId)
+ } else await release(tabId)
owner.delete(tabId)
pages.set((snapshot) => {
if (snapshot[tabId] === undefined) return snapshot
@@ -391,6 +505,8 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
return {
tabs,
pages,
+ holds,
+ setHolds(value) { holds.set(value && typeof value === 'object' ? value : {}) },
/** Restore a Bot's saved strip (or an empty one) so the panel can render it. */
ensureTabs,
openTab,
@@ -450,7 +566,11 @@ export function createBotBrowsers({ bridge, createSource, storage = globalThis.l
dispose() {
disposed = true
clearTimeout(resizeLater)
- for (const entry of views.values()) void release(entry.tabId)
+ for (const entry of views.values()) {
+ if (entry.appView) void bridge.views.place(entry.tabId, rect ?? { left: 0, top: 0, width: 800, height: 600 }, false)
+ else void release(entry.tabId)
+ }
+ appViewEvents?.()
layer?.remove()
},
}
@@ -497,8 +617,10 @@ export function serveBrowserReads({ browsers, post, clientId = globalThis.crypto
const open = browsers.openPages()
reported = JSON.stringify(open)
try {
- const requests = await post('browser-wait', { clientId, open }, poll.signal)
+ const response = await post('browser-wait', { clientId, open }, poll.signal)
+ browsers.setHolds?.(response?.holds)
failures = 0
+ const requests = Array.isArray(response) ? response : response?.requests
for (const request of Array.isArray(requests) ? requests : []) void answer(request)
} catch {
if (stopped) return
diff --git a/ds-bot/src/client/browser-overlap.js b/ds-bot/src/client/browser-overlap.js
new file mode 100644
index 00000000..fb816435
--- /dev/null
+++ b/ds-bot/src/client/browser-overlap.js
@@ -0,0 +1,4 @@
+export function rectsOverlap(a, b) {
+ return a.width > 0 && a.height > 0 && b.width > 0 && b.height > 0
+ && a.left < b.right && a.right > b.left && a.top < b.bottom && a.bottom > b.top
+}
diff --git a/ds-bot/src/client/browser-pane.js b/ds-bot/src/client/browser-pane.js
index 09c1b2cc..a273d4dd 100644
--- a/ds-bot/src/client/browser-pane.js
+++ b/ds-bot/src/client/browser-pane.js
@@ -15,6 +15,7 @@ import { SideChatView } from './side-chat.js'
import { panelHasStrip, sideChatsOf, stripMenuKeys } from './side-model.js'
import { DotsIcon, FolderIcon, SideIcon, TerminalIcon } from './icons.js'
import { t } from './i18n.js'
+import { rectsOverlap } from './browser-overlap.js'
export const PANEL_LINGER_MS = 300
const WIDTH_KEY = STORAGE.browserWidth
@@ -284,11 +285,12 @@ function PanelStrip({ bot, subject, view, ids, active, sides, side, canBrowse, r
// The browser view: toolbar (back, forward, reload/stop, address) over the frame
// the guest webview covers.
-function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages }) {
+function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages, hold }) {
const page = useBrowserPages(pages => pages[tabId ?? '']) ?? EMPTY_PAGE
const frame = useRef(null)
const input = useRef(null)
const [draft, setDraft] = useState(null)
+ const [covered, setCovered] = useState(false)
// Cookie imports (the desktop app's bridge only): 'menu' lists the sources; a card
// stage then runs running → preview → result | error above the guest.
@@ -306,7 +308,21 @@ function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages })
// The guest goes on screen once the panel has landed and settled on its width,
// and only over a page: the empty, error and import states are the panel's own.
- const showGuest = landed && !leaving && Boolean(page.url) && !page.error && flow === null
+ const showGuest = landed && !leaving && !covered && Boolean(page.url) && !page.error && flow === null
+
+ useEffect(() => {
+ const update = () => {
+ const panel = frame.current?.getBoundingClientRect()
+ const overlaps = panel && [...document.querySelectorAll('[role="dialog"],[role="menu"],[role="listbox"]')]
+ .some(element => rectsOverlap(panel, element.getBoundingClientRect()))
+ setCovered(Boolean(overlaps))
+ }
+ const observer = new MutationObserver(update)
+ observer.observe(document.body, { subtree: true, childList: true, attributes: true, attributeFilter: ['role', 'data-leaving'] })
+ window.addEventListener('resize', update)
+ update()
+ return () => { observer.disconnect(); window.removeEventListener('resize', update) }
+ }, [])
// Leaving the tab or the panel abandons an unconfirmed import.
useEffect(() => {
@@ -390,6 +406,13 @@ function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages })
const editing = draft !== null
const secure = /^https:/i.test(page.url)
return h('div', { className: 'bt-panel-view bt-panel-browser', 'aria-label': t("{name}'s browser", { name: bot.name }) },
+ actions.browserTakeover ? h('div', { className: 'bt-browser-takeover', 'data-held': hold?.by === 'user' || undefined },
+ h('span', null, hold?.by === 'user' ? t('You control this browser') : hold?.by === 'bot' ? t('The Bot handed this browser to you: {reason}', { reason: hold.reason }) : t('The Bot controls this browser')),
+ hold?.by
+ ? h('button', { type: 'button', onClick: () => void actions.browserGiveBack(bot.id) }, t('Give back'))
+ : h('span', { className: 'bt-browser-takeover-actions' },
+ h('button', { type: 'button', onClick: () => void actions.browserTakeover(bot.id) }, t('Take over')),
+ h('button', { type: 'button', onClick: () => void actions.browserStop(bot.id) }, t('Stop')))) : null,
h('div', { className: 'bt-browser-bar' },
h('button', { type: 'button', className: 'bt-icon-btn', 'aria-label': t('Back'), title: t('Back'), disabled: !page.canGoBack, onClick: () => actions.browserCommand(tabId, 'back') }, h(BackIcon)),
h('button', { type: 'button', className: 'bt-icon-btn', 'aria-label': t('Forward'), title: t('Forward'), disabled: !page.canGoForward, onClick: () => actions.browserCommand(tabId, 'forward') }, h(ForwardIcon)),
@@ -438,6 +461,7 @@ function BrowserView({ bot, tabId, leaving, landed, actions, useBrowserPages })
: null))
}
+
function importError(error) {
if (error === 'too-large') return t('That file is too large to be a cookie export.')
if (error === 'unreadable') return t('That file could not be read.')
@@ -539,7 +563,7 @@ function ImportCard({ flow, page, busy, onApply, onCancel, onCloseAndImport, onD
* Session id the panel follows and `side` the open chat's id. Without the
* Desktop bridge the browser view never shows; side chats need no bridge.
*/
-export function BotPanel({ pane, leaving, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages }) {
+export function BotPanel({ pane, leaving, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages, useBrowserHolds }) {
const bot = roster.byId[pane.id]
const canBrowse = Boolean(bot && roster.browser === true && actions.browserShow)
const sideChats = bot ? sideChatsOf(roster, bot.id) : []
@@ -548,6 +572,7 @@ export function BotPanel({ pane, leaving, roster, actions, useSessions, useSessi
const tabState = useBrowserTabs(map => (bot ? (map[bot.id] ?? EMPTY_TABS) : EMPTY_TABS))
const active = tabState.active
const page = useBrowserPages(pages => (active ? pages[active] : null) ?? EMPTY_PAGE)
+ const hold = useBrowserHolds?.(map => bot ? map[bot.id] ?? null : null) ?? null
const root = useRef(null)
const fly = useRef(null)
const [landed, setLanded] = useState(false)
@@ -750,7 +775,7 @@ export function BotPanel({ pane, leaving, roster, actions, useSessions, useSessi
dragging ? createPortal(h('div', { className: 'bt-drag-shield', 'aria-hidden': true }), document.body) : null,
hasStrip ? h(PanelStrip, { bot, subject: pane.id, view, ids: tabState.ids, active, sides: sideChats, side: pane.side, canBrowse, reading, onView: switchView, actions: panelActions, useBrowserPages }) : null,
view === 'browser' && bot
- ? h(BrowserView, { key: `browser:${active}`, bot, tabId: active, leaving, landed: landed && !settling, actions, useBrowserPages })
+ ? h(BrowserView, { key: `browser:${active}`, bot, tabId: active, leaving, landed: landed && !settling, actions, useBrowserPages, hold })
: view === 'side' && sideChat
? h('div', { key: `side:${sideChat.id}`, className: 'bt-panel-view' }, h(SideChatView, { subject: pane.id, chat: sideChat, actions: panelActions }))
: h('div', { key: `details:${pane.id}:${pane.renaming ? 'rename' : ''}:${pane.tab ?? ''}`, className: 'bt-panel-view' },
@@ -797,6 +822,10 @@ export const BROWSER_CSS = `
.bt-panel[data-settling]>.bt-ptabs,.bt-panel[data-settling]>.bt-panel-view{flex-shrink:0;align-self:flex-end;box-sizing:border-box;width:calc(var(--bt-panel-w,320px) - .5px)}
.bt-panel-browser{position:relative}
.bt-browser-bar{position:relative;display:flex;align-items:center;gap:2px;height:52px;padding:0 10px 0 12px;flex:none;border-bottom:.5px solid var(--bt-line)}
+.bt-browser-takeover{display:flex;align-items:center;justify-content:space-between;gap:10px;padding:8px 12px;background:var(--bt-hover);border-bottom:.5px solid var(--bt-line);font-size:12px;color:var(--bt-ink-2)}
+.bt-browser-takeover[data-held]{background:var(--bt-active)}
+.bt-browser-takeover-actions{display:flex;gap:6px}
+.bt-browser-takeover button{border:1px solid var(--bt-line-2);border-radius:8px;padding:5px 9px;background:var(--bt-main);color:var(--bt-ink);font:inherit;cursor:pointer}
.bt-browser-bar .bt-icon-btn:disabled{opacity:.35;cursor:default;background:none}
.bt-browser-address{flex:1;min-width:0;display:flex;align-items:center;gap:6px;height:32px;margin:0 4px;padding:0 12px;border-radius:999px;background:var(--bt-hover);border:.5px solid transparent;transition:background-color .12s ease,border-color .12s ease,box-shadow .12s ease}
.bt-browser-address:hover{background:var(--bt-active)}
diff --git a/ds-bot/src/client/i18n.js b/ds-bot/src/client/i18n.js
index c150b147..1e95cdb3 100644
--- a/ds-bot/src/client/i18n.js
+++ b/ds-bot/src/client/i18n.js
@@ -266,6 +266,12 @@ const ZH = {
'cookies.txt, JSON, storageState': 'cookies.txt、JSON、storageState',
'Copy of its cookies': '复制它的 cookie',
'No browsers found': '没有找到浏览器',
+ 'The Bot controls this browser': 'Bot 正在控制此浏览器',
+ 'The Bot is waiting for you': 'Bot 正在等待你',
+ 'You control this browser': '你正在控制此浏览器',
+ 'The Bot handed this browser to you: {reason}': 'Bot 已将此浏览器交给你:{reason}',
+ 'Take over': '接管',
+ 'Give back': '交还',
'{name} is running': '{name} 正在运行',
'It must close for a moment so its cookies can be copied. It will reopen with your tabs.': '需要先把它关掉一下才能复制 cookie,之后会自动重新打开并恢复你的标签页。',
'Close {name} and import': '关闭 {name} 并导入',
diff --git a/ds-bot/src/client/index.js b/ds-bot/src/client/index.js
index 49dc509a..bc9c817a 100644
--- a/ds-bot/src/client/index.js
+++ b/ds-bot/src/client/index.js
@@ -542,6 +542,9 @@ export function apply(ctx) {
browserNavigate: (tabId, input) => browsers.navigate(tabId, input),
browserCommand: (tabId, name) => browsers.command(tabId, name),
browserFocus: (tabId) => browsers.focus(tabId),
+ browserTakeover: bridge.views ? botId => call('browser-takeover', { botId }) : undefined,
+ browserStop: bridge.views ? botId => call('browser-stop', { botId }) : undefined,
+ browserGiveBack: bridge.views ? botId => call('browser-give-back', { botId }) : undefined,
// Cookie imports, only where the bridge offers them (the desktop app): a file or an
// installed browser's profile is previewed, then applied into the Bot's storage key.
browserImport: typeof bridge.importSources === 'function' ? {
@@ -600,7 +603,8 @@ export function apply(ctx) {
actions.openOwnApp = async () => { await ownAppUpdates?.open?.() }
const browserPages = browsers?.pages ?? createSource({})
const browserTabs = browsers?.tabs ?? createSource({})
- const face = () => ({ actions, hooks: { roster, ui, activity, work, memoryNews, registry, turnClock, live, cards, heldCards, browserPages, browserTabs, surface, update } })
+ const browserHolds = browsers?.holds ?? createSource({})
+ const face = () => ({ actions, hooks: { roster, ui, activity, work, memoryNews, registry, turnClock, live, cards, heldCards, browserPages, browserTabs, browserHolds, surface, update } })
ctx.effect(() => noticeApprovals({
work,
roster,
diff --git a/ds-bot/src/client/overlays.js b/ds-bot/src/client/overlays.js
index f50adff9..67d7450b 100644
--- a/ds-bot/src/client/overlays.js
+++ b/ds-bot/src/client/overlays.js
@@ -14,7 +14,7 @@ import { useLinger } from './motion.js'
const DIALOG_EXIT_MS = 160
const MENU_EXIT_MS = 120
-export function Overlays({ useRoster, useUi, useSessions, useSessionStatus, useActivity, useWork, useRegistry, useBrowserPages, useBrowserTabs, useUpdate, actions }) {
+export function Overlays({ useRoster, useUi, useSessions, useSessionStatus, useActivity, useWork, useRegistry, useBrowserPages, useBrowserTabs, useBrowserHolds, useUpdate, actions }) {
const roster = useRoster(value => value)
const ui = useUi(value => value)
useRegistry(value => value)
@@ -47,7 +47,7 @@ export function Overlays({ useRoster, useUi, useSessions, useSessionStatus, useA
const owner = memory ? memoryOwner(memory) : null
const [menu, menuLeaving] = useLinger(ui.menu, MENU_EXIT_MS)
return h(Fragment, null,
- pane ? createPortal(h(BotPanel, { pane, leaving: open === null, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages }), document.body) : null,
+ pane ? createPortal(h(BotPanel, { pane, leaving: open === null, roster, actions, useSessions, useSessionStatus, useActivity, useWork, useBrowserTabs, useBrowserPages, useBrowserHolds }), document.body) : null,
ui.newChat ? createPortal(h(NewChat, { key: ui.newChat, mode: ui.newChat, roster, actions }), document.body) : null,
ui.palette ? createPortal(h(CommandPalette, { key: ui.palette.token, start: ui.palette.query, token: ui.palette.token, roster, actions, useSessions }), document.body) : null,
ui.exchange ? createPortal(h(ExchangeDialog, { pair: ui.exchange, roster, actions }), document.body) : null,
diff --git a/ds-bot/src/host/browser-plugins.js b/ds-bot/src/host/browser-plugins.js
index fee8f137..2e1779e3 100644
--- a/ds-bot/src/host/browser-plugins.js
+++ b/ds-bot/src/host/browser-plugins.js
@@ -1,6 +1,7 @@
// The whole team uses one browser plugin. Each Bot still gets its own profile
// directory, including the group Sessions where that Bot speaks.
import { join } from 'node:path'
+import { randomUUID } from 'node:crypto'
import { defineTool } from '@deepseek-ai/dsh-tools'
import { completeDeepseek } from './browser/deepseek.js'
import { mountBrowserPlugin } from './browser/mount.js'
@@ -9,6 +10,27 @@ import * as script from './browser/script.js'
import * as task from './browser/task.js'
export const browserPlugins = { refs, script, task }
+function appBrowserEndpoint(botId) {
+ if (typeof process.send !== 'function') throw new Error('The app browser relay is unavailable.')
+ const id = randomUUID()
+ return new Promise((resolve, reject) => {
+ const timeout = setTimeout(() => {
+ process.off('message', receive)
+ reject(new Error('The app browser relay did not respond.'))
+ }, 10_000)
+ const receive = message => {
+ if (message?.type !== 'ds-bot:browser-endpoint-response' || message.id !== id) return
+ clearTimeout(timeout)
+ process.off('message', receive)
+ if (typeof message.url === 'string') resolve(message.url)
+ else reject(new Error(message.error || 'The app browser relay is unavailable.'))
+ }
+ process.on('message', receive)
+ process.send({ type: 'ds-bot:browser-endpoint', id, botId }, error => {
+ if (error) receive({ type: 'ds-bot:browser-endpoint-response', id, error: error.message })
+ })
+ })
+}
const NAMES = Object.keys(browserPlugins)
const GRANT_KINDS = new Set(['post', 'upload', 'download', 'delete'])
@@ -97,6 +119,9 @@ export function install(rt) {
headless,
playwrightUrl,
executablePath,
+ connect: process.env.DS_BOT_APP === '1' && typeof process.send === 'function' ? appBrowserEndpoint : undefined,
+ getHold: botId => rt.browserChannel?.holdOf(botId) ?? null,
+ onHold: (botId, hold) => rt.browserChannel?.setHold(botId, hold),
authorize: (exec, botId, action) => authorizeBrowserAction(rt, exec, botId, action),
complete: rt.config.browserComplete ?? completeDeepseek,
async keyOf(exec) {
diff --git a/ds-bot/src/host/browser.js b/ds-bot/src/host/browser.js
index d5236e80..c2910093 100644
--- a/ds-bot/src/host/browser.js
+++ b/ds-bot/src/host/browser.js
@@ -26,7 +26,9 @@ export function createBrowserChannel({ now = Date.now, waitMs = WAIT_MS, staleMs
const clients = new Map()
/** @type {Map<string, {client:any, resolve:(value:any) => void, timer:any}>} */
const pending = new Map()
+ const holds = new Map()
let nextId = 0
+ let holdVersion = 0
const present = client => client.wake !== undefined || now() - client.seen < staleMs
// The most recently seen window that shows this Bot's browser.
@@ -55,12 +57,16 @@ export function createBrowserChannel({ now = Date.now, waitMs = WAIT_MS, staleMs
* `waitMs`. A newer poll from the same window ends the older one empty.
*/
function wait(clientId, open) {
- const client = clients.get(clientId) ?? { open: {}, seen: 0, queue: [] }
+ const client = clients.get(clientId) ?? { open: {}, seen: 0, queue: [], holdVersion: 0 }
clients.set(clientId, client)
client.open = open && typeof open === 'object' ? open : {}
client.seen = now()
client.end?.()
const ready = due(client)
+ if (client.holdVersion !== holdVersion) {
+ client.holdVersion = holdVersion
+ return Promise.resolve(ready)
+ }
if (ready.length > 0) return Promise.resolve(ready)
const resend = client.queue.length === 0 ? waitMs
: Math.max(0, Math.min(...client.queue.map(item => item.sentAt + resendMs - now())))
@@ -69,6 +75,7 @@ export function createBrowserChannel({ now = Date.now, waitMs = WAIT_MS, staleMs
clearTimeout(timer)
if (client.end === end) client.wake = client.end = undefined
client.seen = now()
+ client.holdVersion = holdVersion
resolve(items)
}
const end = () => finish([])
@@ -122,13 +129,25 @@ export function createBrowserChannel({ now = Date.now, waitMs = WAIT_MS, staleMs
return pages
}
+ function setHold(botId, hold) {
+ if (hold === null) holds.delete(botId)
+ else holds.set(botId, { by: hold.by, reason: hold.reason, note: hold.note })
+ holdVersion += 1
+ for (const client of clients.values()) client.wake?.()
+ return holds.get(botId) ?? null
+ }
+
+ const holdOf = botId => holds.get(botId) ?? null
+ const holdSnapshot = () => Object.fromEntries(holds)
+
function dispose() {
for (const client of clients.values()) client.end?.()
for (const id of [...pending.keys()]) settle(id, { error: 'DS Bot stopped before the page was read.' })
clients.clear()
+ holds.clear()
}
- return { wait, result, read, openPages, dispose }
+ return { wait, result, read, openPages, setHold, holdOf, holdSnapshot, dispose }
}
const clean = (value, max) => String(value ?? '').replace(/\s+/g, ' ').trim().slice(0, max)
diff --git a/ds-bot/src/host/browser/mount.js b/ds-bot/src/host/browser/mount.js
index 5fc4abfb..fc2d5b2a 100644
--- a/ds-bot/src/host/browser/mount.js
+++ b/ds-bot/src/host/browser/mount.js
@@ -20,7 +20,7 @@ export const browserOutput = {
* @param {any} ctx
* @param {{ toolDefs: any[], defineTool: Function, keyOf: (exec: any) => Promise<string|undefined>, profileRoot: string, downloadRoot?: string, headless: boolean, playwrightUrl?: string, executablePath?: string, actionTimeout?: number, authorize?: (exec: any, botId: string, action: any) => Promise<void>, complete?: Function }} options
*/
-export function mountBrowserPlugin(ctx, { toolDefs, defineTool, keyOf, profileRoot, downloadRoot, headless, playwrightUrl, executablePath, actionTimeout, authorize, complete }) {
+export function mountBrowserPlugin(ctx, { toolDefs, defineTool, keyOf, profileRoot, downloadRoot, headless, playwrightUrl, executablePath, actionTimeout, authorize, complete, connect, getHold, onHold }) {
const sessions = new Map()
ctx.effect(() => async () => {
const open = [...sessions.values()]
@@ -47,6 +47,9 @@ export function mountBrowserPlugin(ctx, { toolDefs, defineTool, keyOf, profileRo
playwrightUrl,
executablePath,
actionTimeout,
+ connect,
+ getHold: () => getHold?.(botId),
+ onHold: hold => onHold?.(botId, hold),
})
sessions.set(botId, session)
}
diff --git a/ds-bot/src/host/browser/session.js b/ds-bot/src/host/browser/session.js
index bd039f64..7ccd5093 100644
--- a/ds-bot/src/host/browser/session.js
+++ b/ds-bot/src/host/browser/session.js
@@ -87,27 +87,35 @@ async function loadChromium(playwrightUrl) {
/**
* @param {{ botId: string, profileRoot: string, headless?: boolean, playwrightUrl?: string, executablePath?: string, actionTimeout?: number }} options
*/
-export async function openBotBrowser({ botId, profileRoot, downloadDir, headless = true, playwrightUrl, executablePath, actionTimeout = ACTION_TIMEOUT }) {
+export async function openBotBrowser({ botId, profileRoot, downloadDir, headless = true, playwrightUrl, executablePath, actionTimeout = ACTION_TIMEOUT, connect, getHold, onHold }) {
const userDataDir = join(profileRoot, botId)
- await mkdir(userDataDir, { recursive: true })
const chromium = await loadChromium(playwrightUrl)
- const context = await chromium.launchPersistentContext(userDataDir, {
- headless,
- viewport: { width: 1280, height: 900 },
- acceptDownloads: true,
- ...(executablePath ? { executablePath } : {}),
- args: ['--disable-dev-shm-usage'],
- })
+ const browser = connect
+ ? await chromium.connectOverCDP(await connect(botId))
+ : undefined
+ if (!connect) await mkdir(userDataDir, { recursive: true })
+ const context = browser
+ ? browser.contexts()[0]
+ : await chromium.launchPersistentContext(userDataDir, {
+ headless,
+ viewport: { width: 1280, height: 900 },
+ acceptDownloads: true,
+ ...(executablePath ? { executablePath } : {}),
+ args: ['--disable-dev-shm-usage'],
+ })
+ if (!context) throw new Error('The app browser relay has no browser context.')
const page = context.pages()[0] ?? await context.newPage()
const cookieFile = join(userDataDir, 'cookies.json')
- try {
- const saved = JSON.parse(await readFile(cookieFile, 'utf8'))
- if (Array.isArray(saved) && saved.length > 0) await context.addCookies(saved)
- } catch { /* no cookies yet */ }
- return createSession({ context, page, botId, userDataDir, cookieFile, downloadDir, actionTimeout })
+ if (!browser) {
+ try {
+ const saved = JSON.parse(await readFile(cookieFile, 'utf8'))
+ if (Array.isArray(saved) && saved.length > 0) await context.addCookies(saved)
+ } catch { /* no cookies yet */ }
+ }
+ return createSession({ context, browser, page, botId, userDataDir, cookieFile, downloadDir, actionTimeout, getHold, onHold })
}
-function createSession({ context, page, botId, userDataDir, cookieFile, downloadDir, actionTimeout }) {
+function createSession({ context, browser, page, botId, userDataDir, cookieFile, downloadDir, actionTimeout, getHold, onHold }) {
const pages = new Set()
const unknownByOrigin = new Map()
const unknownPageOrigins = new WeakMap()
@@ -120,6 +128,9 @@ function createSession({ context, page, botId, userDataDir, cookieFile, download
let pendingDelete = null
const ready = () => {
+ const externalHold = getHold?.()
+ if (externalHold?.by === 'user') throw new Error('The user has taken over this browser. Wait for them to give it back.')
+ if (hold && getHold && externalHold == null) hold = null
if (hold) throw new Error(`The browser is waiting for the user (${hold.reason}). Ask them to ${hold.note || 'take over'}, then call browser_resume.`)
}
@@ -600,10 +611,13 @@ function createSession({ context, page, botId, userDataDir, cookieFile, download
handoff(reason, note) {
const kind = REASONS.has(reason) ? reason : 'other'
hold = { reason: kind, note: String(note ?? '').slice(0, 300) }
+ onHold?.({ by: 'bot', reason: kind, note: hold.note })
return `Waiting for the user (${kind}). ${hold.note || 'Ask them to take over.'} Call browser_resume when they have finished.`
},
resume() {
+ if (getHold?.()?.by === 'user') throw new Error('The user has taken over this browser. Only the user can give it back.')
hold = null
+ onHold?.(null)
return 'The browser is yours again.'
},
async runScript(source) {
@@ -643,6 +657,7 @@ function createSession({ context, page, botId, userDataDir, cookieFile, download
return finalAnswer === undefined ? 'Script finished without done().' : String(finalAnswer)
},
async close() {
+ if (browser) { await browser.close().catch(() => {}); return }
const cookies = await context.cookies().catch(() => [])
const kept = cookies.map(cookie => cookie.expires > 0 ? cookie : { ...cookie, expires: Math.floor(Date.now() / 1000) + 30 * 24 * 60 * 60 })
await writeFile(cookieFile, JSON.stringify(kept)).catch(() => {})
diff --git a/ds-bot/src/host/channel.js b/ds-bot/src/host/channel.js
index 6dad5c69..83064f3f 100644
--- a/ds-bot/src/host/channel.js
+++ b/ds-bot/src/host/channel.js
@@ -458,11 +458,44 @@ export function install(rt) {
const clientId = typeof body.clientId === 'string' ? body.clientId.slice(0, 100) : ''
if (clientId === '') return fail('A browser poll needs a clientId', 'bot/invalid')
await rt.load()
- return ok(await rt.browserChannel.wait(clientId, rt.browserOpen(body.open)))
+ const requests = await rt.browserChannel.wait(clientId, rt.browserOpen(body.open))
+ return ok({ requests, holds: rt.browserChannel.holdSnapshot() })
}
case 'browser-result':
if (rt.browserChannel === undefined) return fail('Bot browsers are off on this host', 'bot/browser-off')
return ok(rt.browserChannel.result(body))
+ case 'browser-takeover': {
+ if (rt.browserChannel === undefined) return fail('Bot browsers are off on this host', 'bot/browser-off')
+ await rt.load()
+ const botId = String(body.botId ?? '')
+ if (!rt.botOf(botId)) return fail('Unknown Bot', 'bot/not-found')
+ rt.browserChannel.setHold(botId, { by: 'user', reason: 'takeover', note: 'The user is controlling this browser.' })
+ return ok(true)
+ }
+ case 'browser-give-back': {
+ if (rt.browserChannel === undefined) return fail('Bot browsers are off on this host', 'bot/browser-off')
+ await rt.load()
+ const botId = String(body.botId ?? '')
+ if (!rt.botOf(botId)) return fail('Unknown Bot', 'bot/not-found')
+ if (rt.browserChannel.holdOf(botId) === null) return fail('This browser is not held', 'bot/forbidden')
+ rt.browserChannel.setHold(botId, null)
+ return ok(true)
+ }
+ case 'browser-stop': {
+ await rt.load()
+ const botId = String(body.botId ?? '')
+ if (!rt.botOf(botId)) return fail('Unknown Bot', 'bot/not-found')
+ const sessionId = rt.chatOf(botId)
+ if (sessionId && rt.live.get(sessionId)?.turn !== undefined) {
+ const agent = await rt.resolveAgent(sessionId)
+ agent.cancel({ kind: 'user' })
+ }
+ for (const room of Object.values(rt.state.rooms ?? {})) {
+ if (room.members?.includes(botId)) await rt.stopMemberWork?.(room.id, botId)
+ }
+ rt.browserChannel?.setHold(botId, { by: 'user', reason: 'stopped', note: 'The user is controlling this browser.' })
+ return ok(true)
+ }
case 'diagnostics':
await rt.load()
return ok(await rt.diagnostics.report(body))
diff --git a/ds-bot/test/browser-pane.test.js b/ds-bot/test/browser-pane.test.js
new file mode 100644
index 00000000..55675326
--- /dev/null
+++ b/ds-bot/test/browser-pane.test.js
@@ -0,0 +1,11 @@
+import assert from 'node:assert/strict'
+import { test } from 'node:test'
+import { rectsOverlap } from '../src/client/browser-overlap.js'
+
+test('overlay coverage requires positive rectangle intersection', () => {
+ const panel = { left: 10, top: 20, right: 110, bottom: 120, width: 100, height: 100 }
+ assert.equal(rectsOverlap(panel, { left: 50, top: 60, right: 150, bottom: 160, width: 100, height: 100 }), true)
+ assert.equal(rectsOverlap(panel, { left: 110, top: 20, right: 150, bottom: 60, width: 40, height: 40 }), false)
+ assert.equal(rectsOverlap(panel, { left: 200, top: 200, right: 250, bottom: 250, width: 50, height: 50 }), false)
+ assert.equal(rectsOverlap(panel, { left: 50, top: 60, right: 50, bottom: 90, width: 0, height: 30 }), false)
+})
diff --git a/ds-bot/test/browser-plugins.test.js b/ds-bot/test/browser-plugins.test.js
index a4a833b9..2c4b8b51 100644
--- a/ds-bot/test/browser-plugins.test.js
+++ b/ds-bot/test/browser-plugins.test.js
@@ -99,6 +99,36 @@ test('no browser plugin is mounted unless one is selected for the team', (t) =>
assert.equal(ctx.registeredTools.has('browser_task'), false)
})
+test('user browser controls are not exposed as Bot tools', (t) => {
+ const { ctx, stop } = start({ browser: true, browserPlugin: 'refs' })
+ t.after(stop)
+ for (const name of ['browser-takeover', 'browser-give-back', 'browser-stop']) {
+ assert.equal(ctx.registeredTools.has(name), false)
+ }
+})
+
+test('browser-stop cancels the Bot chat, stops its room work, and sets a user hold', async (t) => {
+ const { rt, call, stop } = start({ browser: true })
+ t.after(stop)
+ await rt.load()
+
+ rt.state.rooms.room1 = { id: 'room1', members: ['b2'], sessions: { b2: 'room-session' } }
+ rt.state.rooms.room2 = { id: 'room2', members: ['m1'], sessions: { m1: 'other-session' } }
+ rt.live.set('b2', { turn: {} })
+ rt.live.set('room-session', { turn: {} })
+ const cancelled = []
+ const stoppedRooms = []
+ rt.resolveAgent = async sessionId => ({ cancel: reason => cancelled.push({ sessionId, reason }) })
+ rt.stopMemberWork = async (roomId, botId) => { stoppedRooms.push({ roomId, botId }) }
+
+ assert.equal((await call('browser-stop', { botId: 'b2' })).value, true)
+ assert.deepEqual(cancelled, [{ sessionId: 'b2', reason: { kind: 'user' } }])
+ assert.deepEqual(stoppedRooms, [{ roomId: 'room1', botId: 'b2' }])
+ assert.deepEqual(rt.browserChannel.holdOf('b2'), {
+ by: 'user', reason: 'stopped', note: 'The user is controlling this browser.',
+ })
+})
+
test('the selected plugin is the same for every Bot, and profiles stay apart', async (t) => {
const seen = []
const { ctx, stop } = start({
diff --git a/ds-bot/test/install.test.js b/ds-bot/test/install.test.js
index b71d0845..5def2c4f 100644
--- a/ds-bot/test/install.test.js
+++ b/ds-bot/test/install.test.js
@@ -205,7 +205,9 @@ test('in the Desktop profile read_browser is the last tool and reads through the
const poll = call('browser-wait', { clientId: 'w1', open: { b2: { url: 'https://example.com/', title: 'Example' }, nope: { url: 'https://x/' } } })
await new Promise(resolve => setTimeout(resolve, 10))
const reading = read({})
- const requests = (await poll).value
+ const response = (await poll).value
+ const requests = response.requests
+ assert.deepEqual(response.holds, {})
assert.deepEqual(requests.map(request => request.botId), ['b2'])
assert.deepEqual((await call('browser-result', { id: requests[0].id, ok: true, page: { url: 'https://example.com/', title: 'Example', text: 'Hello from the page', items: [{ role: 'link', name: 'More', href: 'https://example.com/more' }] } })).value, true)
const text = await reading
@@ -215,6 +217,20 @@ test('in the Desktop profile read_browser is the last tool and reads through the
assert.equal((await call('state', {})).value.browser, true)
})
+test('browser takeover and give-back are user-only and publish hold state', async t => {
+ const previous = process.env.DS_BOT_APP
+ process.env.DS_BOT_APP = '1'
+ t.after(() => { if (previous === undefined) delete process.env.DS_BOT_APP; else process.env.DS_BOT_APP = previous })
+ const { call, stop } = start(TEAM, { services: { profileContext: { name: 'ds-bot' } } })
+ t.after(stop)
+ await call('state')
+ assert.equal((await call('browser-takeover', { botId: 'b2' })).value, true)
+ const poll = (await call('browser-wait', { clientId: 'hold-window', open: {} })).value
+ assert.deepEqual(poll.holds.b2, { by: 'user', reason: 'takeover', note: 'The user is controlling this browser.' })
+ assert.equal((await call('browser-give-back', { botId: 'b2' })).value, true)
+ assert.deepEqual((await call('browser-wait', { clientId: 'hold-window', open: {} })).value.holds, {})
+})
+
test('the desktop app has read_browser too; DSH Web does not', (t) => {
process.env.DS_BOT_APP = '1'
t.after(() => { delete process.env.DS_BOT_APP })