Skip to content

armgit / cindy

Public Imported from GitHub
0

Consider it done. The open-source AI agent that works out of the box · 想到,就能做到。开源、开箱即用的 AI Agent。

fix(remote-desktop): 主动恢复操作权与输入异常

Dash05bc7d2cabfe
diff --git a/apps/desktop/src/main/remote-desktop/__tests__/captureBoundary.test.ts b/apps/desktop/src/main/remote-desktop/__tests__/captureBoundary.test.ts
index 8a8f724a35..839e447497 100644
--- a/apps/desktop/src/main/remote-desktop/__tests__/captureBoundary.test.ts
+++ b/apps/desktop/src/main/remote-desktop/__tests__/captureBoundary.test.ts
@@ -189,8 +189,8 @@ vi.mock('../controller', () => ({
this.stop();
}
stopAndRestore = h.stopAndRestore;
- releaseControl() {
- h.releaseControl();
+ releaseControl(recoverInput = false) {
+ h.releaseControl(recoverInput);
}
tick() {}
input = h.input;
@@ -554,6 +554,7 @@ it('turns an input-helper failure into a control release instead of a session st
// Input is a lease-scoped capability: the desktop session keeps its lease,
// its capture owner and its media when the helper dies.
expect(h.releaseControl).toHaveBeenCalledTimes(1);
+ expect(h.releaseControl).toHaveBeenLastCalledWith(true);
expect(h.stop).not.toHaveBeenCalled();
expect(h.owner.dead).toBe(false);
});
@@ -566,6 +567,7 @@ it('releases control when the input host refuses a batch before injecting it', (
// later take-control must actually restart the helper.
expect(() => h.deps.input([{ kind: 'release' }])).toThrow('DESKTOP_INPUT_UNAVAILABLE');
expect(h.releaseControl).toHaveBeenCalledTimes(1);
+ expect(h.releaseControl).toHaveBeenLastCalledWith(true);
expect(h.stop).not.toHaveBeenCalled();
});
diff --git a/apps/desktop/src/main/remote-desktop/__tests__/controller.test.ts b/apps/desktop/src/main/remote-desktop/__tests__/controller.test.ts
index 409bfcdf0c..e6a7c25831 100644
--- a/apps/desktop/src/main/remote-desktop/__tests__/controller.test.ts
+++ b/apps/desktop/src/main/remote-desktop/__tests__/controller.test.ts
@@ -1551,6 +1551,89 @@ describe('remote desktop authority and lifecycle', () => {
expect(h.deps.stopVideo).not.toHaveBeenCalled();
expect(h.controller.hasLease(lease.lease)).toBe(true);
});
+ // Unit regression: native failure must repair this lease's input immediately,
+ // without a viewer heartbeat. Existing release tests only checked teardown.
+ it.each([false, true])(
+ 'repairs native input once, renewed only by fresh input (%s)',
+ async (freshInput) => {
+ const h = harness();
+ const { lease } = await h.start();
+ await h.controller.request('phone', { op: 'control', lease, enabled: true });
+ h.controller.input(lease, 1, [{ kind: 'key', code: 'KeyA', down: true }]);
+ await h.controller.releaseControl(true);
+ expect(h.controller.state).toEqual({ peer: 'phone', controlling: true });
+ expect(h.deps.startInput).toHaveBeenCalledTimes(2);
+ expect(h.deps.input).toHaveBeenCalledTimes(1);
+ expect(h.deps.stopVideo).not.toHaveBeenCalled();
+ // A helper that immediately fails again must not form a restart loop;
+ // fresh input renews the opportunity to repair a later independent failure.
+ if (freshInput) h.controller.input(lease, 2, [{ kind: 'release' }]);
+ await h.controller.releaseControl(true);
+ expect(h.controller.state?.controlling).toBe(freshInput);
+ expect(h.deps.startInput).toHaveBeenCalledTimes(freshInput ? 3 : 2);
+ },
+ );
+ it('does not restart recursively when native recovery fails during startup', async () => {
+ const h = harness();
+ const { lease } = await h.start();
+ await h.controller.request('phone', { op: 'control', lease, enabled: true });
+ vi.mocked(h.deps.startInput).mockImplementationOnce(async () => {
+ await h.controller.releaseControl(true);
+ throw new Error('DESKTOP_INPUT_UNAVAILABLE');
+ });
+ await h.controller.releaseControl(true);
+ expect(h.controller.state?.controlling).toBe(false);
+ expect(h.deps.startInput).toHaveBeenCalledTimes(2);
+ expect(h.deps.stopVideo).not.toHaveBeenCalled();
+ });
+ it.each(['presentation', 'stop', 'takeover', 'revoked', 'control'])(
+ 'does not restore failed input after a newer %s decision',
+ async (ending) => {
+ const h = harness();
+ const { lease } = await h.start();
+ await h.controller.request('phone', { op: 'control', lease, enabled: true });
+ const recovery = h.controller.releaseControl(true);
+ if (ending === 'stop') h.controller.stopByUser();
+ else if (ending === 'revoked') h.revoke();
+ else if (ending === 'takeover')
+ await h.controller.request('other', { op: 'start', displayId: '1', takeover: true });
+ else
+ await h.controller.request('phone', {
+ op: ending,
+ lease,
+ enabled: ending === 'presentation',
+ });
+ await recovery;
+ expect(h.deps.startInput).toHaveBeenCalledTimes(1);
+ expect(h.controller.state?.controlling ?? false).toBe(false);
+ },
+ );
+ it.each([false, true])(
+ 'preserves safety settings during input repair, clearing only on failure (%s)',
+ async (failed) => {
+ const h = harness();
+ h.deps.privacyScreen = vi.fn(async () => {});
+ h.deps.stopPrivacyScreen = vi.fn();
+ h.deps.hostMute = vi.fn(async () => {});
+ h.deps.stopHostMute = vi.fn(async () => {});
+ h.deps.clipboardVersion = vi.fn(async () => '1');
+ h.deps.stopClipboardVersion = vi.fn();
+ const { lease } = await h.start();
+ await h.controller.request('phone', { op: 'control', lease, enabled: true });
+ await h.controller.request('phone', { op: 'privacyScreen', lease, enabled: true });
+ await h.controller.request('phone', { op: 'hostMute', lease, enabled: true });
+ await h.controller.request('phone', { op: 'clipboardSync', lease, enabled: true });
+ if (failed)
+ vi.mocked(h.deps.startInput).mockRejectedValueOnce(
+ new Error('DESKTOP_INPUT_UNAVAILABLE'),
+ );
+ await h.controller.releaseControl(true);
+ expect(h.deps.stopPrivacyScreen).toHaveBeenCalledTimes(failed ? 1 : 0);
+ expect(h.deps.stopHostMute).toHaveBeenCalledTimes(failed ? 1 : 0);
+ expect(h.deps.stopClipboardVersion).toHaveBeenCalledTimes(failed ? 1 : 0);
+ expect(h.controller.state?.controlling).toBe(!failed);
+ },
+ );
});
it.each(['control', 'presentation', 'failure', 'stop'] as const)(
'releases input before failing safety restoration on %s, without stopping replacement input',
diff --git a/apps/desktop/src/main/remote-desktop/controller.ts b/apps/desktop/src/main/remote-desktop/controller.ts
index f4cfa27ad2..c60e0a70c8 100644
--- a/apps/desktop/src/main/remote-desktop/controller.ts
+++ b/apps/desktop/src/main/remote-desktop/controller.ts
@@ -237,12 +237,14 @@ export class RemoteDesktopController {
/** Revoke input synchronously; optional OS restoration must never retain it
* or stop a replacement helper after an asynchronous restore settles.
*/
- private revokeControl(): Promise<void> {
+ private revokeControl(preserveSafety = false): Promise<void> {
if (this.active) this.active.controlling = false;
this.controlGeneration++;
this.clipboardTransfer.reset();
this.deps.stopInput();
- const restoring = this.clearSafety();
+ // Repairing just the input helper must not expose a privacy-masked desktop
+ // or unmute the host without the viewer seeing a settings change.
+ const restoring = preserveSafety ? Promise.resolve() : this.clearSafety();
this.deps.changed();
return restoring;
}
@@ -250,6 +252,7 @@ export class RemoteDesktopController {
private lastFrame = -Infinity;
private controlGeneration = 0;
private inputStarting = false;
+ private inputRecoveryUsed = false;
private userStopped = new Map<string, string>();
private lastEnded: { peer: string; lease: string } | null = null;
constructor(private readonly deps: DesktopControllerDeps) {}
@@ -580,21 +583,61 @@ export class RemoteDesktopController {
* control bit, so release control and keep everything else: ending the lease
* here would tear down capture and media, and every phone tap would surface as
* a reconnect even though the desktop session itself is healthy. The viewer
- * observes the new state on its next heartbeat or input attempt. A pending
- * start is also cancelled so a failed helper cannot restore control later.
+ * can ask again if recovery fails. Native failures opt into one immediate
+ * restart; layout changes only release until their geometry is confirmed.
+ * A pending start is cancelled, never recursively restarted.
*/
- releaseControl(): void {
+ async releaseControl(recoverInput = false): Promise<void> {
const active = this.active;
if (!active || (!active.controlling && !this.inputStarting)) return;
- void this.revokeControl().catch(() => log.warn('Safety restoration failed after control loss'));
+ const recover =
+ recoverInput &&
+ active.controlling &&
+ !this.inputStarting &&
+ !this.inputRecoveryUsed &&
+ !this.starting &&
+ !this.displayChanging &&
+ !active.backgroundViewing;
+ if (recover) this.inputRecoveryUsed = true;
+ try {
+ const restoring = this.revokeControl(recover);
+ const generation = this.controlGeneration;
+ await restoring;
+ // A takeover can be awaiting capabilities before it changes the lease or
+ // generation. Its explicit decision takes precedence over this repair.
+ if (
+ !recover ||
+ this.starting ||
+ this.displayChanging ||
+ this.active !== active ||
+ generation !== this.controlGeneration
+ )
+ return;
+ this.require(active.peer, active.lease);
+ const restarting = this.takeControl(active.peer, active, true);
+ const restartGeneration = this.controlGeneration;
+ try {
+ await restarting;
+ } catch (error) {
+ if (this.active === active && restartGeneration === this.controlGeneration)
+ await this.revokeControl();
+ throw error;
+ }
+ } catch {
+ // The viewer retains its picture and can retry via the normal permission
+ // path. No recursive restart if the replacement helper also fails.
+ log.debug('Input recovery or safety restoration deferred');
+ }
}
/** Starts input for the lease's current display; shared by `control` and `autoControl`. */
private async takeControl(
peer: string,
active: NonNullable<RemoteDesktopController['active']>,
+ recovering = false,
): Promise<void> {
if (this.locking) throw new Error('DESKTOP_BUSY');
if (this.inputStarting) throw new Error('DESKTOP_INPUT_BUSY');
+ if (!recovering) this.inputRecoveryUsed = false;
this.setBackgroundViewing(active, false);
this.clipboardTransfer.reset();
const generation = ++this.controlGeneration;
@@ -696,7 +739,11 @@ export class RemoteDesktopController {
if (!active.controlling) throw new Error('DESKTOP_VIEW_ONLY');
if (sequence <= active.sequence) return; // never replay a click after retries/reconnect
active.sequence = sequence;
+ const generation = this.controlGeneration;
this.deps.input(request.events);
+ // Fresh user activity permits another repair, but an idle crash-looping
+ // helper cannot repeatedly restart itself after its ready handshake.
+ if (generation === this.controlGeneration) this.inputRecoveryUsed = false;
}
async request(peer: string, raw: unknown): Promise<unknown> {
const request = parseRemoteDesktopRequest(raw);
diff --git a/apps/desktop/src/main/remote-desktop/index.ts b/apps/desktop/src/main/remote-desktop/index.ts
index 64d382cd34..ce742384a9 100644
--- a/apps/desktop/src/main/remote-desktop/index.ts
+++ b/apps/desktop/src/main/remote-desktop/index.ts
@@ -268,8 +268,10 @@ let pending: {
timer: ReturnType<typeof setTimeout>;
} | null = null;
// A dead input helper or a refused injection is an input failure, not a session
-// failure: release control and keep the lease, capture and media running.
-const input = new DesktopInputHost(() => remoteDesktop.releaseControl());
+// failure: repair input once while keeping the lease, capture and media running.
+const input = new DesktopInputHost(() => {
+ void remoteDesktop.releaseControl(true);
+});
const clipboardCounter = new ClipboardCounter(resolveDesktopInputBinary);
// A display change holds native capture instead of stopping it: frames pause
// until the changed display is known, then the same stream follows it. The
@@ -916,7 +918,7 @@ export const remoteDesktop: RemoteDesktopController = new RemoteDesktopControlle
// lease's display is unavailable). Release control so the host and the
// viewer agree, and so taking control again genuinely restarts the
// helper instead of being skipped as "already controlling".
- remoteDesktop.releaseControl();
+ void remoteDesktop.releaseControl(true);
throw error;
}
},
diff --git a/apps/mobile/src/remote-desktop/RemoteDesktopScreen.tsx b/apps/mobile/src/remote-desktop/RemoteDesktopScreen.tsx
index b589622e27..068e562a9f 100644
--- a/apps/mobile/src/remote-desktop/RemoteDesktopScreen.tsx
+++ b/apps/mobile/src/remote-desktop/RemoteDesktopScreen.tsx
@@ -342,7 +342,6 @@ export function RemoteDesktopSession({
[]);
const [viewerReadyRevision, setViewerReadyRevision] = useState(0);
const active = useRef<RemoteDesktopLease | null>(null);
- const wantsControl = useRef(true);
// View only is a local guard against stray touches. The host keeps control
// and its input helper, so switching never waits for the computer.
const [viewOnlySelected, setViewOnlySelected] = useState(false);
@@ -355,8 +354,9 @@ export function RemoteDesktopSession({
const pendingHostControl = useRef<boolean | null>(null);
const controlInFlight = useRef(false);
const controlRestoration = useRef<Promise<void> | null>(null);
- // Revoking background permission must not lose the pending fullscreen input restore.
- const restoreAfterPipDisabled = useRef(false);
+ // A foreground event can arrive while control/settings are still settling.
+ // Finish that restore as soon as the existing operation settles.
+ const foregroundRestorePending = useRef(false);
const finishBackgroundTransition = useRef<(() => void) | null>(null);
const recovery = useRef({
enabled: true,
@@ -850,7 +850,7 @@ export function RemoteDesktopSession({
presentation.current = false;
actualPresentation.current = false;
pipPrepared.current = false;
- restoreAfterPipDisabled.current = false;
+ foregroundRestorePending.current = false;
presentationGeneration.current++;
settingInFlight.current = false;
pendingPresentation.current?.(false);
@@ -975,8 +975,8 @@ export function RemoteDesktopSession({
* The host owns the control bit. When it reports that this viewer no longer
* controls — input injection failed, the input helper went away, or control
* was retracted — drop to view only instead of rebuilding the session: the
- * lease, the picture and the existing view-only hint all survive, and taking
- * control again retries from where the user is.
+ * lease and picture survive. Known input failures restore control immediately;
+ * the existing heartbeat retries missed or failed foreground recovery.
*/
const releaseControl = useCallback(() => {
const current = active.current;
@@ -996,10 +996,11 @@ export function RemoteDesktopSession({
* anything else rebuilds the session. Sitting next to the two actions it
* chooses between keeps the three call sites from drifting apart.
*/
- const resolveControlFailure = (cause: unknown) => {
+ const resolveControlFailure = (cause: unknown, restoreInput = false) => {
switch (controlFailureAction(cause)) {
case "release":
releaseControl();
+ if (restoreInput) restoreInlinePresentationRef.current();
break;
case "ignore":
break;
@@ -1012,7 +1013,6 @@ export function RemoteDesktopSession({
controlling: boolean,
) => {
pendingHostControl.current = null;
- wantsControl.current = controlling;
if (current.controlling !== controlling) {
current.controlling = controlling;
if (!controlling) {
@@ -1033,9 +1033,10 @@ export function RemoteDesktopSession({
enabled: boolean,
) => {
if (controlInFlight.current) return;
+ if (enabled) foregroundRestorePending.current = false;
controlInFlight.current = true;
pendingHostControl.current = enabled;
- return request<{ controlling: boolean }>({
+ const operation = request<{ controlling: boolean }>({
op: "control",
lease: current.lease,
enabled,
@@ -1049,15 +1050,27 @@ export function RemoteDesktopSession({
resolveControlFailure(cause);
})
.finally(() => {
- if (active.current === current) controlInFlight.current = false;
+ if (active.current !== current) return;
+ controlInFlight.current = false;
+ if (foregroundRestorePending.current) restoreInlinePresentationRef.current();
});
+ // PiP must wait for both recovery grants and overflow releases before it
+ // authorizes background viewing; an older operation cannot clear a new one.
+ controlRestoration.current = operation;
+ void operation.finally(() => {
+ if (controlRestoration.current === operation) controlRestoration.current = null;
+ });
+ return operation;
};
const requestHostControlRef = useRef(requestHostControl);
const restoreInlinePresentation = () => {
const current = active.current;
if (
!current ||
- (!pipPrepared.current && !restoreAfterPipDisabled.current) ||
+ (current.controlling &&
+ !pipPrepared.current &&
+ !foregroundRestorePending.current) ||
+ !capsRef.current?.value.canControl ||
presentation.current ||
actualPresentation.current ||
!focusedRef.current ||
@@ -1065,33 +1078,27 @@ export function RemoteDesktopSession({
AppState.currentState !== "active"
)
return;
+ if (
+ connecting.current ||
+ controlInFlight.current ||
+ settingInFlight.current ||
+ pendingHostControl.current === false
+ ) {
+ foregroundRestorePending.current = true;
+ return;
+ }
pipPrepared.current = false;
- restoreAfterPipDisabled.current = false;
+ foregroundRestorePending.current = false;
send({
type: "pipPolicy",
enabled: pipEnabledRef.current && frameReady,
authorized: false,
});
- // Returning to a visible viewer restores the user's previous input mode
- // through the host's normal authorization path, never by a local override.
- const restoration = wantsControl.current
- ? requestHostControlRef.current(current, true)
- : request({
- op: "presentation",
- lease: current.lease,
- enabled: false,
- })
- .then(() => {})
- .catch((cause) => {
- if (active.current === current) fail(cause);
- });
- if (restoration) {
- controlRestoration.current = restoration;
- void restoration.finally(() => {
- if (controlRestoration.current === restoration)
- controlRestoration.current = null;
- });
- }
+ // Foreground always retains host control; view only guards input locally.
+ // A temporary release/refusal must not replace that intent. App/route/PiP
+ // events restore immediately; heartbeats retry when a transition was busy
+ // or its request failed, using the same lease and host permission checks.
+ requestHostControlRef.current(current, true);
};
const restoreInlinePresentationRef = useRef(restoreInlinePresentation);
restoreInlinePresentationRef.current = restoreInlinePresentation;
@@ -1132,7 +1139,7 @@ export function RemoteDesktopSession({
takeover,
...(handoff ? { handoff } : {}),
// Hosts with `autoControl` grant control with the lease.
- control: wantsControl.current,
+ control: true,
onStart: () => {
recovery.current.resuming = true;
},
@@ -1239,7 +1246,7 @@ export function RemoteDesktopSession({
// Entering remote desktop is the user's intent to control. The existing
// host permission and ownership gates still decide whether it is allowed.
if (next.controlling) send({ type: "control", enabled: inputAllowed(true) });
- else if (result.canControl && wantsControl.current) {
+ else if (result.canControl) {
try {
const control = await viewerSession.current!.control(true);
if (current !== generation.current) return;
@@ -1256,7 +1263,10 @@ export function RemoteDesktopSession({
} catch (cause) {
if (current === generation.current) fail(cause);
} finally {
- if (current === generation.current) connecting.current = false;
+ if (current === generation.current) {
+ connecting.current = false;
+ if (foregroundRestorePending.current) restoreInlinePresentationRef.current();
+ }
}
},
[
@@ -1482,22 +1492,32 @@ export function RemoteDesktopSession({
// The host owns the control bit. Follow it when it retracts control;
// when a local transition timed out, retry a release or restore the
// local bit so a held key cannot stay down behind a view-only phone.
- if (active.current !== current || presentation.current) return;
+ if (
+ active.current !== current ||
+ presentation.current ||
+ actualPresentation.current ||
+ !focusedRef.current ||
+ AppState.currentState !== "active"
+ )
+ return;
if (result.controlling === false) {
- // startInput can still be settling while this heartbeat was in
- // flight. Clearing a pending take-control here would leave later
- // host-true beats with nothing to restore after a lost reply.
- if (pendingHostControl.current === true) return;
+ // Do not consume an intent while its request is settling. Once it
+ // settles, host-false is a reason to retry, not to remain stuck.
+ if (controlInFlight.current) return;
pendingHostControl.current = null;
if (current.controlling) releaseControl();
- return;
- }
- if (pendingHostControl.current === false) {
+ } else if (pendingHostControl.current === false) {
requestHostControlRef.current(current, false);
return;
- }
- if (!current.controlling && pendingHostControl.current === true)
+ } else if (
+ result.controlling === true &&
+ !controlInFlight.current &&
+ !current.controlling &&
+ pendingHostControl.current === true
+ ) {
applyConfirmedControl(current, true);
+ }
+ restoreInlinePresentationRef.current();
})
.catch((cause) => {
if (active.current !== current) return;
@@ -2160,7 +2180,9 @@ export function RemoteDesktopSession({
// posts control:false, which clears that release without flushing it.
// Tell the host to drop control (stopInput still injects a native
// release) so a held key/button cannot stay down while we view only.
+ if (!current.controlling) break;
releaseControl();
+ foregroundRestorePending.current = true;
requestHostControl(current, false);
break;
}
@@ -2218,7 +2240,7 @@ export function RemoteDesktopSession({
})()
.catch((cause) => {
if (!ownsBatch()) return;
- resolveControlFailure(cause);
+ resolveControlFailure(cause, true);
})
.finally(() => {
if (!ownsBatch()) return;
@@ -2267,7 +2289,6 @@ export function RemoteDesktopSession({
presentation.current = false;
send({ type: "presentation", enabled: false });
}
- wantsControl.current = true;
if (pendingHostControl.current === false) {
// Overflow timed out with an unconfirmed host release. Taking control
// first would skip stopInput while a key/button may still be held.
@@ -2356,6 +2377,8 @@ export function RemoteDesktopSession({
}
} finally {
settingInFlight.current = false;
+ if (active.current === current && foregroundRestorePending.current)
+ restoreInlinePresentationRef.current();
if (alive.current) setSettingsRevision((value) => value + 1);
}
};
@@ -2527,6 +2550,7 @@ export function RemoteDesktopSession({
finishBackgroundTransition.current = null;
if (presentationGeneration.current === transition)
settingInFlight.current = false;
+ if (currentTransition()) restoreInlinePresentationRef.current();
if (alive.current) setSettingsRevision((value) => value + 1);
}
};
@@ -2566,7 +2590,7 @@ export function RemoteDesktopSession({
presentationGeneration.current++;
send({ type: "pipPolicy", enabled: false });
send({ type: "presentation", enabled: false });
- restoreAfterPipDisabled.current ||= pipPrepared.current;
+ foregroundRestorePending.current ||= pipPrepared.current;
pipPrepared.current = false;
const current = active.current;
if (current)
@@ -2762,6 +2786,8 @@ export function RemoteDesktopSession({
} finally {
settingInFlight.current = false;
setSettingBusy(false);
+ if (active.current === current && foregroundRestorePending.current)
+ restoreInlinePresentationRef.current();
}
};
// The host restores its own mode when a viewer leaves. Reapply this phone's
@@ -2796,8 +2822,7 @@ export function RemoteDesktopSession({
if (
earlyDisplayFailed.current ||
rememberedDisplayGaveUp.current ||
- !hostCaps.canControl ||
- !wantsControl.current
+ !hostCaps.canControl
)
return false;
const remembered = await readRememberedResolution(
diff --git a/apps/mobile/src/remote-desktop/__tests__/screen.test.tsx b/apps/mobile/src/remote-desktop/__tests__/screen.test.tsx
index a4558d1aa0..f832fed3dc 100644
--- a/apps/mobile/src/remote-desktop/__tests__/screen.test.tsx
+++ b/apps/mobile/src/remote-desktop/__tests__/screen.test.tsx
@@ -670,7 +670,7 @@ describe("remote desktop controls", () => {
return { ...callbacks, render };
};
it.each(["unsupported", "no-frame", "control-busy", "settings-busy"])(
- "ends a detached native lease when PiP admission is refused: %s",
+ "settles detached native PiP admission without stranding the lease: %s",
async (reason) => {
fixture.nativeMedia = true;
fixture.pipEnabled = true;
@@ -737,6 +737,15 @@ describe("remote desktop controls", () => {
expect(finish).toBeDefined();
}
await act(async () => render(false));
+ if (reason === "control-busy") {
+ expect(onEnded).not.toHaveBeenCalled();
+ expect(requests().filter((r) => r.op === "presentation")).toHaveLength(0);
+ await act(async () => finish?.());
+ await nativePresentation("presentation", { active: true });
+ expect(requests().filter((r) => r.op === "presentation" && r.enabled)).toHaveLength(1);
+ expect(requests().filter((r) => r.op === "stop")).toHaveLength(0);
+ return;
+ }
expect(onEnded).toHaveBeenCalledOnce();
expect(requests().filter((r) => r.op === "stop")).toHaveLength(1);
expect(requests().find((r) => r.op === "stop")).toMatchObject({
@@ -1038,6 +1047,130 @@ describe("remote desktop controls", () => {
expect(button("viewOnly").getAttribute("aria-selected")).toBe("true");
expect(visibleInputHint()).toBe("remoteDesktop.viewOnlyHint");
});
+ // Regression: a failed fullscreen restore consumed the PiP flags, and later
+ // host-false heartbeats never retried. Exercise the real screen with an
+ // in-memory transport (unit); existing PiP tests only covered a granted restore.
+ it.each([
+ "INVOKE_TIMEOUT",
+ "DESKTOP_INPUT_BUSY",
+ "DESKTOP_INPUT_UNAVAILABLE",
+ "refused",
+ ])(
+ "recovers fullscreen input on the existing heartbeat after %s",
+ async (failure) => {
+ await retainedViewer();
+ const original = fixture.invoke.getMockImplementation()!;
+ let attempts = 0;
+ fixture.invoke.mockImplementation((...args) => {
+ const req = args[2][0];
+ if (req.op === "heartbeat")
+ return Promise.resolve({ controlling: attempts > 1 });
+ if (req.op === "control" && req.enabled && ++attempts === 1)
+ return failure === "refused"
+ ? Promise.resolve({ controlling: false })
+ : Promise.reject(new Error(failure));
+ return original(...args);
+ });
+ await nativePresentation("presentation", { active: false });
+ expect(attempts).toBe(1);
+ await act(async () => vi.advanceTimersByTimeAsync(3000));
+ expect(attempts).toBe(2);
+ expect(
+ sent()
+ .filter((m) => m.type === "control")
+ .at(-1),
+ ).toEqual({
+ type: "control",
+ enabled: true,
+ });
+ expect(requests().filter((r) => r.op === "start")).toHaveLength(1);
+ expect(requests().filter((r) => r.op === "stop")).toHaveLength(0);
+ },
+ );
+ it("does not recover input in PiP, but recovers in fullscreen without overriding view only", async () => {
+ await retainedViewer();
+ const original = fixture.invoke.getMockImplementation()!;
+ fixture.invoke.mockImplementation((...args) =>
+ args[2][0].op === "heartbeat"
+ ? Promise.resolve({ controlling: false })
+ : original(...args),
+ );
+ const controls = () =>
+ requests().filter((r) => r.op === "control" && r.enabled).length;
+ const before = controls();
+ await act(async () => vi.advanceTimersByTimeAsync(6000));
+ expect(controls()).toBe(before);
+ act(() => button("operations").click());
+ await act(async () => button("viewOnly").click());
+ await nativePresentation("presentation", { active: false });
+ await act(async () => vi.advanceTimersByTimeAsync(3000));
+ expect(controls()).toBeGreaterThan(before);
+ expect(button("viewOnly").getAttribute("aria-selected")).toBe("true");
+ expect(
+ sent()
+ .filter((m) => m.type === "control")
+ .at(-1)?.enabled,
+ ).toBe(false);
+ });
+ it.each(["DESKTOP_STOPPED", "DESKTOP_ACCESSIBILITY_PERMISSION"])(
+ "respects a terminal host refusal during fullscreen recovery: %s",
+ async (code) => {
+ await retainedViewer();
+ const original = fixture.invoke.getMockImplementation()!;
+ fixture.invoke.mockImplementation((...args) =>
+ args[2][0].op === "control"
+ ? Promise.reject(new Error(code))
+ : original(...args),
+ );
+ await nativePresentation("presentation", { active: false });
+ const attempts = requests().filter((r) => r.op === "control").length;
+ await act(async () => vi.advanceTimersByTimeAsync(30_000));
+ expect(requests().filter((r) => r.op === "control")).toHaveLength(
+ attempts,
+ );
+ expect(requests().filter((r) => r.op === "start")).toHaveLength(1);
+ },
+ );
+ it("serializes heartbeat recovery with a new Home transition and ignores its late reply after exit", async () => {
+ await retainedViewer();
+ const original = fixture.invoke.getMockImplementation()!;
+ let attempts = 0;
+ let finish!: (value: unknown) => void;
+ fixture.invoke.mockImplementation((...args) => {
+ const req = args[2][0];
+ if (req.op === "heartbeat")
+ return Promise.resolve({ controlling: false });
+ if (req.op === "control" && req.enabled) {
+ if (++attempts === 1)
+ return Promise.reject(new Error("INVOKE_TIMEOUT"));
+ return new Promise((resolve) => {
+ finish = resolve;
+ });
+ }
+ return original(...args);
+ });
+ await nativePresentation("presentation", { active: false });
+ await act(async () => vi.advanceTimersByTimeAsync(9000));
+ expect(attempts).toBe(2);
+ const grants = () =>
+ requests().filter((r) => r.op === "presentation" && r.enabled).length;
+ const before = grants();
+ await act(async () => {
+ AppState.currentState = "inactive";
+ fixture.appState!("inactive");
+ });
+ await nativePresentation("presentation", { active: true });
+ expect(grants()).toBe(before);
+ act(() => button("operations").click());
+ await act(async () => button("disconnect").click());
+ const messages = sent().length;
+ await act(async () => finish({ controlling: true }));
+ expect(grants()).toBe(before);
+ expect(sent().slice(messages)).not.toContainEqual({
+ type: "control",
+ enabled: true,
+ });
+ });
it("waits for actual PiP before navigating back and keeps the connection mounted", async () => {
const viewer = await retainedViewer();
await nativePresentation("presentation", { active: false });
@@ -1271,6 +1404,31 @@ describe("remote desktop controls", () => {
);
},
);
+ it("restores fullscreen input as soon as an overlapping sound change settles", async () => {
+ fixture.systemAudio = true;
+ await retainedViewer();
+ const original = fixture.invoke.getMockImplementation()!;
+ let finish!: () => void;
+ fixture.invoke.mockImplementation((...args) =>
+ args[2][0].op === "presentation" && !args[2][0].enabled
+ ? new Promise((resolve) => {
+ finish = () => resolve({});
+ })
+ : original(...args),
+ );
+ act(() => button("operations").click());
+ await act(async () => button("sound").click());
+ const controls = () =>
+ requests().filter((r) => r.op === "control" && r.enabled).length;
+ const before = controls();
+ await nativePresentation("presentation", { active: false });
+ expect(controls()).toBe(before);
+ await act(async () => finish());
+ // No timer advance: the settings completion resumes the deferred restore.
+ expect(controls()).toBe(before + 1);
+ expect(visibleInputHint()).toBe("remoteDesktop.touchHint");
+ expect(requests().filter((r) => r.op === "stop")).toHaveLength(0);
+ });
it("preserves authorized native media while fullscreen signaling reconnects", async () => {
const viewer = await retainedViewer();
const controls = requests().filter((r) => r.op === "control").length;
@@ -1545,12 +1703,13 @@ describe("remote desktop controls", () => {
const first = channelRequest().id;
await viewer({ type: "channelRequestState", id: first, sent: true });
await viewer({ type: "fallback", reason: "failed" });
- // Control is free again right away and the next release uses the relay;
- // the request the channel took is never sent twice.
- await release();
- expect(relayedControl().filter((r) => r.enabled === false)).toHaveLength(
- 1,
- );
+ // The sent release has an unknown outcome. Reconcile before granting
+ // control, and never replay that release onto a different transport.
+ const original = fixture.invoke.getMockImplementation()!;
+ fixture.invoke.mockImplementation((...args) => args[2][0].op === "heartbeat"
+ ? Promise.resolve({ controlling: false }) : original(...args));
+ await act(async () => vi.advanceTimersByTimeAsync(3000));
+ expect(relayedControl()).toEqual([{ op: "control", lease: "lease", enabled: true }]);
expect(channelRequest().id).toBe(first);
});
@@ -1559,7 +1718,7 @@ describe("remote desktop controls", () => {
await live();
await release();
expect(sent().some((m) => m.type === "channelRequest")).toBe(false);
- expect(relayedControl()).toHaveLength(1);
+ expect(relayedControl().map((r) => r.enabled)).toEqual([false, true]);
});
it("prefers the native receiver on iOS and falls back to the WebView", async () => {
@@ -1593,6 +1752,7 @@ describe("remote desktop controls", () => {
request: { op: "control", lease: "lease", enabled: false },
});
expect(sent().some((m) => m.type === "channelRequest")).toBe(false);
+ fixture.nativeRequest.mockResolvedValue(false);
await act(async () =>
fixture.nativeMessage!({
nativeEvent: {
@@ -1606,10 +1766,8 @@ describe("remote desktop controls", () => {
},
}),
);
- fixture.nativeRequest.mockResolvedValue(false);
- // Leaving view only asks the host again once it no longer controls.
- await act(async () => button("viewOnly").click());
- await act(async () => button("viewOnly").click());
+ // Reacquisition follows the acknowledged release immediately, and uses
+ // the WebView when native transport can no longer accept the request.
expect(channelRequest()).toMatchObject({
request: { op: "control", lease: "lease", enabled: true },
});
@@ -1735,7 +1893,9 @@ describe("remote desktop controls", () => {
fixture.invoke.mockImplementation((...args) =>
args[2][0].op === "heartbeat"
? Promise.resolve({ controlling: false })
- : original(...args),
+ : args[2][0].op === "control"
+ ? Promise.reject(new Error("DESKTOP_INPUT_UNAVAILABLE"))
+ : original(...args),
);
await act(async () => vi.advanceTimersByTimeAsync(3100));
expect(
@@ -4427,8 +4587,11 @@ describe("remote desktop controls", () => {
expect(host.textContent).not.toContain("remoteDesktop.reconnecting");
expect(sent()).toContainEqual({ type: "control", enabled: false });
act(() => button("operations").click());
- expect(visibleInputHint()).toBe("remoteDesktop.controlUnavailableHint");
expect(button("viewOnly").getAttribute("aria-selected")).toBe("false");
+ // Recovery is triggered by the rejection, without advancing a heartbeat.
+ expect(visibleInputHint()).toBe("remoteDesktop.touchHint");
+ // Recovery grants fresh control; it never replays the rejected click.
+ expect(requests().filter((r) => r.op === "input")).toHaveLength(1);
});
it("keeps control and the session when an input reply is lost", async () => {
await connect();
@@ -4457,8 +4620,10 @@ describe("remote desktop controls", () => {
expect(requests().filter((r) => r.op === "stop")).toHaveLength(0);
expect(host.textContent).not.toContain("remoteDesktop.reconnecting");
});
- it("stops input without changing the user's view-only choice when host control is revoked", async () => {
+ it("restores the chosen input mode automatically when host control is revoked", async () => {
await connect();
+ act(() => button("operations").click());
+ act(() => button("pointer").click());
const original = fixture.invoke.getMockImplementation()!;
fixture.invoke.mockImplementation((...args) =>
args[2][0].op === "heartbeat"
@@ -4469,8 +4634,7 @@ describe("remote desktop controls", () => {
expect(sent()).toContainEqual({ type: "control", enabled: false });
expect(requests().filter((r) => r.op === "stop")).toHaveLength(0);
expect(host.textContent).not.toContain("remoteDesktop.reconnecting");
- act(() => button("operations").click());
- expect(visibleInputHint()).toBe("remoteDesktop.controlUnavailableHint");
+ expect(visibleInputHint()).toBe("remoteDesktop.pointerHint");
expect(button("viewOnly").getAttribute("aria-selected")).toBe("false");
});
it("releases a stalled input batch instead of rebuilding the session", async () => {
@@ -4493,6 +4657,9 @@ describe("remote desktop controls", () => {
});
expect(requests().filter((r) => r.op === "stop")).toHaveLength(0);
expect(host.textContent).not.toContain("remoteDesktop.reconnecting");
+ expect(
+ requests().filter((r) => r.op === "control").map((r) => r.enabled),
+ ).toEqual([true, false, true]);
});
it.each(["resolve", "reject", "native fallback"])("isolates renewed control from an old input %s", async (outcome) => {
if (outcome === "native fallback") fixture.nativeMedia = true;
@@ -4571,6 +4738,10 @@ describe("remote desktop controls", () => {
});
/** The host drops control (overflow release); the phone then enters view only. */
const dropHostControl = async () => {
+ const original = fixture.invoke.getMockImplementation()!;
+ fixture.invoke.mockImplementation((...args) =>
+ args[2][0].op === "control" && args[2][0].enabled
+ ? Promise.resolve({ controlling: false }) : original(...args));
await act(async () => {
fixture.message!({
nativeEvent: {
@@ -4580,6 +4751,7 @@ describe("remote desktop controls", () => {
});
act(() => button("operations").click());
await act(async () => button("viewOnly").click());
+ fixture.invoke.mockImplementation(original);
};
it("finishes a timed-out overflow release before taking control again", async () => {
await connect();
diff --git a/docs/remote-desktop.md b/docs/remote-desktop.md
index 71ef6820fe..ec006ddc43 100644
--- a/docs/remote-desktop.md
+++ b/docs/remote-desktop.md
@@ -329,24 +329,40 @@ device/network verification before claiming a measured connection-success gain.
Control is a lease-scoped capability, not the session itself. When the host can
no longer inject input — the native helper died, it reported a failed injection,
-or its write path failed — it releases control and keeps everything else: the
-lease, the capture owner, the video track and the last picture. It does not call
+or its write path failed — it stops the old input helper and immediately attempts
+one repair on the same lease. Successful repair preserves privacy masks, host
+mute and clipboard settings as well as capture, video and the last picture.
+Failure falls back to the existing control-release cleanup. It does not call
`stop()` on the host. A desktop viewer may still end its own lease when it sees
the lost control; the mobile viewer can keep watching.
The host also releases control when its own input path refuses a batch before
injecting anything, so the two sides cannot disagree about who controls: a later
take-control genuinely restarts the helper instead of being skipped as "already
-controlling".
+controlling". Repair uses the normal authorization, display and lease checks;
+stop, takeover, background presentation and layout changes supersede it. A
+replacement helper that fails before new input does not restart itself again;
+fresh input or a new viewer control request permits another attempt. The
+host-side behavior is covered in
+[`controller.test.ts`](../apps/desktop/src/main/remote-desktop/__tests__/controller.test.ts).
The mobile viewer follows the host's control bit instead of rebuilding the session. A
rejected input batch, a failed control request or a heartbeat that reports
-`controlling: false` all drop the phone to view only with the existing view-only
-hint and take-control action; media and lease identity are untouched. A dropped
+`controlling: false` all stop local input while media and lease identity stay
+untouched. In a visible foreground viewer, input rejection and return from PiP
+immediately request control through the host's normal permission checks. If
+control or settings are still settling, their completion continues the pending
+restore without waiting for a timer. The existing 3-second heartbeat is only a
+fallback after a failed or uncertain attempt; a temporary host refusal does not
+change the intent to control. Actual PiP, pending presentation,
+background/inactive state, route blur, offline signaling and in-flight control
+or settings changes defer recovery. The local **View only** choice still blocks
+input after host control returns. A dropped
stalled batch (WebView overflow) also asks the host to drop control: posting
`control:false` to the WebView clears its queued release without flushing it, so
only an explicit host `control enabled:false` (which stops the input helper and
-injects a native release) can let go of a held key or button. If that host
+injects a native release) can let go of a held key or button. Its confirmation
+immediately requests fresh control, dropping the stale queue. If that host
request times out, the viewer keeps the intended control bit and the next
heartbeat retries the release (or restores local control after a lost
take-control reply) instead of ignoring a host-`true` while the phone stays
@@ -359,7 +375,13 @@ a control request that collides with one still settling — do not rebuild the
session: a batch that may have been injected must not be answered with a
release that discards its key-up, and the heartbeat still owns liveness. Errors
that do mean the lease is gone (`DESKTOP_LEASE_EXPIRED`, `DESKTOP_STOPPED`,
-revocation, an unsupported channel) still recover the session as before.
+revocation, an unsupported channel) still follow the existing session failure
+policy; an explicit stop or permission refusal never triggers automatic control
+reacquisition. Recovery never replays old clicks or keys, replaces the lease or
+resets shared device-link connections. Fullscreen restore, transient refusal,
+overflow release ordering and late replies are covered through the real mobile
+screen in
+[`screen.test.tsx`](../apps/mobile/src/remote-desktop/__tests__/screen.test.tsx).
Desktop viewers require control. They request it on every connection and do not
offer a view-only action. If the host refuses control, revokes it later, rejects