Skip to content

A local-first multi-agent app development platform, built on PenguinHarness

feat(web): 服务器浏览器里用输入法打中文也能进页面

lmliheng2f0cd9ff7ea8
diff --git a/changelog/unreleased/2026-11-01-server-browser.md b/changelog/unreleased/2026-11-01-server-browser.md
index b491c46f..9231bda0 100644
--- a/changelog/unreleased/2026-11-01-server-browser.md
+++ b/changelog/unreleased/2026-11-01-server-browser.md
@@ -42,6 +42,6 @@ The page has one page and no tabs: what it exists for is watching a page your ow
The picture, the clicks, the wheel and the keyboard all work. Whether a data-centre IP can sign into some platform, or make its video play, is that platform's risk control and login problem and has nothing to do with this page showing a page. A browser under its default user agent cannot open some sites at all (it lands on a captcha interstitial), and those sites ask for a sign-in in a real desktop browser too.
-## Input methods are not wired up yet
+## What an input method types gets there too
-What travels back is the keyboard: letters, digits, symbols, Enter, Tab and the arrow keys all reach the page. Chinese — anything an input method composes first and then hands over as a whole string — does not, because the composing happens on your side and there is nothing there to catch it yet. That is the next thing to add, not a setting.
+The keyboard travels as keys: letters, digits, symbols, Enter, Tab and the arrow keys, one at a time. A composed character is not a key press, so an invisible field sits over the picture — that is where the composing lands, and when it commits the whole string goes over at once as `insertText`. The keys pressed during composing are no longer forwarded as ordinary keys, which would both send the page the wrong thing and cut the composition short.
diff --git a/changelog/unreleased/2026-11-01-server-browser.zh.md b/changelog/unreleased/2026-11-01-server-browser.zh.md
index 668ec83c..86ba0602 100644
--- a/changelog/unreleased/2026-11-01-server-browser.zh.md
+++ b/changelog/unreleased/2026-11-01-server-browser.zh.md
@@ -42,6 +42,6 @@
画面、点击、滚动、键盘都通。至于用一个数据中心 IP 的浏览器登录某个平台、或者让视频起播——那是那个平台的风控和登录问题,跟这个页面能不能显示无关。一个默认 UA 的浏览器打不开某些站点(会撞上验证码中间页),而这样的站点在真实桌面浏览器里也会要求登录。
-## 输入法还接不上
+## 输入法打的中文也进得去
-传回去的是键盘事件:字母、数字、符号、回车、Tab 和方向键都能到那个页面;中文这类要输入法先组合、再整段交出来的字还传不过去——组合发生在你自己这一侧,那边还没有一个接住它的地方。这是下一步要补的,不是配置问题。
+键盘走的是按键:字母、数字、符号、回车、Tab、方向键都是一键一传。输入法组合出来的字不是按键,所以画面上盖了一个看不见的输入框——它是组合的落点,组合一结束就把整段文本一次送过去(`insertText`)。组合期间那些键不再当成普通按键转发,否则既会把组合键发错,也会把输入法自己打断。
diff --git a/packages/docs/content/server-browser.en.md b/packages/docs/content/server-browser.en.md
index 245e4c1c..14f1595e 100644
--- a/packages/docs/content/server-browser.en.md
+++ b/packages/docs/content/server-browser.en.md
@@ -30,7 +30,7 @@ Opening **Server Browser** in the sidebar is enough: the browser is started when
- The address bar follows a browser's rules: a bare domain like `example.com` opens over `https://`, and Enter opens what you typed. Anything that is not http or https is refused.
- The picture is the page. **Click it to use it**: clicks, double-clicks, right-clicks, drags and the wheel all reach the page, and so does the keyboard — Tab, Enter and the arrow keys included.
-- **Typing with an input method does not get there yet.** What travels is the keyboard: letters, digits, symbols, Enter, Tab and the arrow keys all work, but the step where an input method composes first and hands over the whole string is not wired up, so Chinese and anything else that needs one cannot be typed into the page for now.
+- **Typing with an input method works too.** Letters, digits, symbols, Enter, Tab and the arrow keys travel as keys, and a composed character — Chinese, and anything else an input method hands over as a whole string — travels as that string. There is an invisible field over the picture, and that is where the composing happens.
- The line above the picture shows the page's title, its address, and whether it is still loading; the right-hand end is the size the browser is drawing the page at.
- The page is redrawn at the size of the box on this page. Narrow the window and the page on the server narrows with it — what you see in this area is the viewport, so nothing has to be magnified.
- When the picture stops — the browser was stopped, or the server was restarted — a line and a **Reload** button take its place; the button reconnects.
diff --git a/packages/docs/content/server-browser.zh.md b/packages/docs/content/server-browser.zh.md
index e7cf71c9..fb134928 100644
--- a/packages/docs/content/server-browser.zh.md
+++ b/packages/docs/content/server-browser.zh.md
@@ -30,7 +30,7 @@ description: 由服务器自己跑起来的浏览器,画面实时传到 Web Ap
- 地址栏按浏览器的规则解读:只写域名(如 `example.com`)时以 `https://` 打开,输入框里回车即打开。不是 http 或 https 的地址会被拒绝。
- 画面就是页面本身。**点击画面即可操作它**:单击、双击、右键、拖动和滚轮都会传给那个页面,键盘输入也一样,按 Tab、回车、方向键都可用。
-- **用输入法打字还传不过去。** 传到那个页面的是键盘:字母、数字、符号、回车、Tab 和方向键都可以,但输入法先在自己这一侧组合、再整段交出来的那一步还没接上,所以中文这类需要输入法的字现在打不进去。
+- **用输入法打字也可以**:字母、数字、符号、回车、Tab 和方向键按键盘传过去,中文这类由输入法组合出来的字整段送过去。画面上盖着一个看不见的输入框,就是输入法组合的落点。
- 画面上方一行显示当前页面的标题、地址和它是否还在加载,右侧是浏览器当前绘制页面的尺寸。
- 页面按这个页面的框大小重绘。把窗口拉窄或拉宽,服务器上的页面会跟着变宽变窄——就是你在这块区域里看到的那个尺寸,所以不用放大镜。
- 画面中断(比如浏览器被停掉、或服务器重启了服务)时,中间会出现一句话和一个**刷新**按钮,点它重新连上。
diff --git a/packages/web/src/features/server-browser/server-browser-page.tsx b/packages/web/src/features/server-browser/server-browser-page.tsx
index ae19041c..9bf9c946 100644
--- a/packages/web/src/features/server-browser/server-browser-page.tsx
+++ b/packages/web/src/features/server-browser/server-browser-page.tsx
@@ -277,6 +277,12 @@ function Frame({
}) {
const image = useRef<HTMLImageElement | null>(null);
const surface = useRef<HTMLDivElement | null>(null);
+ /**
+ * The invisible editable element over the picture. An input method has nowhere to compose
+ * without one: Chinese (and anything else that is composed before it is committed) would never
+ * reach the page, because what travels on the keyboard path is a key, not a composed string.
+ */
+ const field = useRef<HTMLTextAreaElement | null>(null);
/** The size the page is currently drawn at, so a resize of the same size is not sent twice. */
const drawnAt = useRef("");
@@ -354,6 +360,10 @@ function Frame({
const key =
(action: "down" | "up") =>
(event: React.KeyboardEvent<HTMLDivElement>): void => {
+ // While an input method is composing, the keys are its business: they are not characters
+ // yet (the composed text arrives as one piece below), and forwarding or swallowing them
+ // here would both send the page the wrong thing and cut the composition short.
+ if (event.nativeEvent.isComposing || event.key === "Process") return;
event.preventDefault();
// Printable keys carry their text so the page receives a real character; everything else is
// a named key the page's own handlers see.
@@ -371,6 +381,25 @@ function Frame({
});
};
+ /**
+ * Text that arrived as a whole — an input method committing, or a character inserted some way
+ * other than a key press. It goes to the page as one `insertText` rather than as keys, and the
+ * element it was composed in is emptied so the same text is not sent twice. What the element
+ * holds is the truth; the composition's own last string is what stands in when the browser has
+ * already taken the text back out of it.
+ */
+ const composed = useCallback(
+ (committed: string): void => {
+ const box = field.current;
+ if (box === null) return;
+ const text = box.value === "" ? committed : box.value;
+ if (text === "") return;
+ box.value = "";
+ send({ type: "text", text });
+ },
+ [send],
+ );
+
return (
<div className="overflow-hidden rounded-lg border border-[--line] bg-[--surface-sunken]">
<div
@@ -417,6 +446,26 @@ function Frame({
</p>
</div>
)}
+ {/* What an input method composes into: invisible, over everything, and the thing a
+ click on the picture focuses. Without it a composed character has nowhere to land
+ and is lost — the keyboard path carries keys, and a composed string is not one. */}
+ <textarea
+ ref={field}
+ aria-label={S.serverBrowser.inputSurface}
+ tabIndex={-1}
+ spellCheck={false}
+ autoComplete="off"
+ autoCorrect="off"
+ autoCapitalize="off"
+ className="absolute inset-0 h-full w-full resize-none overflow-hidden border-0 bg-transparent text-transparent caret-transparent outline-none"
+ onCompositionEnd={(event) => composed(event.data)}
+ // Text that arrives without a composition — an insert from the browser rather than a
+ // key press — is text as well. During a composition it is not yet: that is what
+ // `composed` is for, when it commits.
+ onInput={(event) => {
+ if (!event.nativeEvent.isComposing) composed("");
+ }}
+ />
</>
)}
</div>
diff --git a/packages/web/src/lib/strings-en.ts b/packages/web/src/lib/strings-en.ts
index b9fdb234..181cb83f 100644
--- a/packages/web/src/lib/strings-en.ts
+++ b/packages/web/src/lib/strings-en.ts
@@ -397,6 +397,8 @@ export const en: Strings = {
streamEnded: "The picture stopped; the browser on the server may have shut down.",
/** The frame area's keyboard hint: what a visitor can do with the page. */
hint: "Click the picture to use the page, or just type.",
+ /** The invisible editable element an input method composes into, over the picture. */
+ inputSurface: "Type here; what you type goes to the page",
},
tracePanel: {
diff --git a/packages/web/src/lib/strings.ts b/packages/web/src/lib/strings.ts
index 9a1230f3..a564b1c6 100644
--- a/packages/web/src/lib/strings.ts
+++ b/packages/web/src/lib/strings.ts
@@ -465,6 +465,8 @@ export const zh = {
streamEnded: "画面已中断,浏览器可能已经停止。",
/** The frame area's keyboard hint: what a visitor can do with the page. */
hint: "点击画面可与页面交互,也可以直接输入。",
+ /** The invisible editable element an input method composes into, over the picture. */
+ inputSurface: "在这里打字,字会送到那个页面",
},
/** The Trace dock panel (the current conversation's Trace files). */