Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
fix(mobile): play and seek videos on Android
Elian Doranec51110790c0
diff --git a/.claude/skills/developing-capacitor-mobile/SKILL.md b/.claude/skills/developing-capacitor-mobile/SKILL.mdindex d35663c3ee..8cdd4bf699 100644--- a/.claude/skills/developing-capacitor-mobile/SKILL.md+++ b/.claude/skills/developing-capacitor-mobile/SKILL.md@@ -49,6 +49,8 @@ Consequences that keep tripping people up:- **`TriliumAssetHandler` must not declare stored properties.** The instance it takes over was allocated as a `WebViewAssetHandler`, so subclass storage would be uninitialized; state lives in `TriliumRequestBridge.shared`. A stopped `WKURLSchemeTask` must never be completed (it raises), which is why `stop` removes the task from the pending map at once.- **Both bridges answer GET and HEAD only.** Android's `shouldInterceptRequest` exposes no request body, and iOS follows suit (its scheme task could read `httpBody`) so the platforms stay alike; any other method gets a 405 with `x-trilium-bridge-error`. A request with a body has to go through `standaloneApi.localFetch` (as `server.ts`'s `ajax()` does) or, for a multipart upload, `standaloneApi.localUpload` (as `uploadForm()` does for `upload()` and imports); a raw `fetch`/`$.ajax` POST to `api/` fails on both platforms.- **Neither native response carries a 3xx** (`WebResourceResponse` rejects one, a scheme task cannot redirect), so `native-request-bridge.ts` turns a redirect (the share pages issue them) into a 200 page with a `<meta http-equiv="refresh">`. It also sets `content-length` to the exact body length (except for HEAD), which WebKit's media loader needs.+- **Android WebView applies a request's `Range` to the intercepted stream itself.** Chromium's `AndroidStreamReaderURLLoader` treats the `InputStream` as the whole resource and skips `first_byte_position` bytes before reading, while keeping our status and headers. The worker has already cut the `206` slice, so `LocalRequestBroker` reads the start from its `Content-Range` and `ChunkStream.skip()` passes over that many bytes without reading. Without it every range past byte 0 lands in the wrong place or runs off the end (`ERR_REQUEST_RANGE_NOT_SATISFIABLE`), which breaks seeking and any MP4 whose `moov` index sits at the end. Keep `available()` at 0: that is what makes Chromium skip its own bounds check.+- **Standalone caps `bytes=N-` media responses at 8 MiB** (`maxOpenRangeLength` in `serveContentWithRanges()`, set by `openPartialInt()` when `getPlatform().isStandalone`), because each response is buffered whole on the page and copied through the bridge. The element asks for the next range on its own; ranges with an explicit end are served as asked.- **Three prefix lists must agree**: `isLocalApiRequest()` in `local-bridge.ts`, `LocalRequestBroker.isLocalPath()` in Java and `isLocalApiPath()` in Swift, including the `/share` rule.- **A service worker registered by an older Android build still controls the page.** Its fetches never reach `shouldInterceptRequest`, and with no `attachServiceWorkerBridge()` nothing answers them. Clearing the app's data or reinstalling removes it.- `sw.ts` has a defensive `self.location.protocol === "capacitor:"` guard and must **let `/_trilium_native_http/` requests fall through** to the WebView (see below); keep both when editing its fetch handler.diff --git a/apps/mobile/android/app/src/main/java/org/triliumnotes/trilium/LocalRequestBroker.java b/apps/mobile/android/app/src/main/java/org/triliumnotes/trilium/LocalRequestBroker.javaindex 1032576cda..dc254633d4 100644--- a/apps/mobile/android/app/src/main/java/org/triliumnotes/trilium/LocalRequestBroker.java+++ b/apps/mobile/android/app/src/main/java/org/triliumnotes/trilium/LocalRequestBroker.java@@ -21,6 +21,8 @@ import java.util.concurrent.TimeUnit;import java.util.concurrent.atomic.AtomicInteger;import java.util.function.Consumer;import java.util.function.Function;+import java.util.regex.Matcher;+import java.util.regex.Pattern;/*** Tracks the local API requests {@link TriliumRequestBridge} forwards to the page, from dispatch@@ -164,6 +166,9 @@ final class LocalRequestBroker {if ("content-type".equals(name)) {contentType = header.getValue();}+ if ("content-range".equals(name) && request.status == 206) {+ request.body.startAt(rangeStart(header.getValue()));+ }responseHeaders.put(header.getKey(), header.getValue());}return new InterceptedResponse(request.status, reasonPhrase(request.status), contentType,@@ -272,6 +277,14 @@ final class LocalRequestBroker {return json.replace("\u2028", "\\u2028").replace("\u2029", "\\u2029");}+ /** The first byte position of a {@code Content-Range: bytes S-E/T} value, or 0 if it has none. */+ static long rangeStart(String contentRange) {+ Matcher matcher = CONTENT_RANGE.matcher(contentRange.trim());+ return matcher.find() ? Long.parseLong(matcher.group(1)) : 0;+ }++ private static final Pattern CONTENT_RANGE = Pattern.compile("^bytes\\s+(\\d+)-\\d+/");+/** WebResourceResponse requires a non-empty reason phrase. */static String reasonPhrase(int status) {switch (status) {@@ -335,6 +348,28 @@ final class LocalRequestBroker {private byte[] current;private int position;private boolean finished;+ /** Bytes before the first queued one, which {@link #skip} passes over without reading. */+ private volatile long skippablePrefix;++ /**+ * Marks the body as starting at {@code offset} of the resource. WebView's loader treats an+ * intercepted stream as the whole resource and skips to the requested range start itself,+ * so for a 206 the page already cut, that skip must land on the first queued byte.+ */+ void startAt(long offset) {+ skippablePrefix = offset;+ }++ @Override+ public long skip(long n) throws IOException {+ long prefix = skippablePrefix;+ if (prefix > 0 && n > 0) {+ long skipped = Math.min(n, prefix);+ skippablePrefix = prefix - skipped;+ return skipped;+ }+ return super.skip(n);+ }void append(byte[] chunk) {if (chunk.length > 0) {diff --git a/apps/mobile/android/app/src/test/java/org/triliumnotes/trilium/LocalRequestBrokerTest.java b/apps/mobile/android/app/src/test/java/org/triliumnotes/trilium/LocalRequestBrokerTest.javaindex e8950fa05a..38b6e4c87c 100644--- a/apps/mobile/android/app/src/test/java/org/triliumnotes/trilium/LocalRequestBrokerTest.java+++ b/apps/mobile/android/app/src/test/java/org/triliumnotes/trilium/LocalRequestBrokerTest.java@@ -104,6 +104,48 @@ public class LocalRequestBrokerTest {assertEquals("hello", read(response.body));}+ @Test+ public void letsWebViewSkipToARangeThePageAlreadyCut() throws Exception {+ LocalRequestBroker broker = readyBroker();+ Map<String, String> headers = new HashMap<>();+ headers.put("Range", "bytes=1000-");+ Future<InterceptedResponse> pending = request(broker, null, "https://localhost/api/notes/v/open-partial", headers);+ int id = idOf(awaitScript(0));++ broker.onMessage("{\"type\":\"head\",\"id\":" + id + ",\"status\":206,\"headers\":{"+ + "\"content-range\":\"bytes 1000-1004/5000\"}}");+ InterceptedResponse response = pending.get(WAIT_MS, TimeUnit.MILLISECONDS);+ broker.onFrame(frame(id, "slice"));+ broker.onMessage("{\"type\":\"end\",\"id\":" + id + "}");++ // WebView treats the stream as the whole resource and skips to the range start itself,+ // in a loop, as SkipToRequestedRange in Chromium's input_stream_reader.cc does.+ assertEquals(0, response.body.available());+ long toSkip = 1000;+ while (toSkip > 0) {+ long skipped = response.body.skip(toSkip);+ assertTrue("the skip must make progress", skipped > 0);+ toSkip -= skipped;+ }+ assertEquals("slice", read(response.body));+ }++ @Test+ public void skipsRealBytesWhenTheResponseIsNotARange() throws Exception {+ LocalRequestBroker broker = readyBroker();+ Future<InterceptedResponse> pending = request(broker, null, "https://localhost/api/notes/x", new HashMap<>());+ int id = idOf(awaitScript(0));++ broker.onMessage("{\"type\":\"head\",\"id\":" + id + ",\"status\":200,\"headers\":{"+ + "\"Content-Range\":\"bytes 0-4/5\"}}");+ InterceptedResponse response = pending.get(WAIT_MS, TimeUnit.MILLISECONDS);+ broker.onFrame(frame(id, "whole"));+ broker.onMessage("{\"type\":\"end\",\"id\":" + id + "}");++ assertEquals(2, response.body.skip(2));+ assertEquals("ole", read(response.body));+ }+@Testpublic void dispatchesAtOnceWhenReadyAndDecodesBase64Chunks() throws Exception {LocalRequestBroker broker = broker(WAIT_MS, false);diff --git a/packages/trilium-core/src/routes/api/files.ts b/packages/trilium-core/src/routes/api/files.tsindex 0cc02cd6fc..44eb2fc805 100644--- a/packages/trilium-core/src/routes/api/files.ts+++ b/packages/trilium-core/src/routes/api/files.ts@@ -8,6 +8,7 @@ import type BAttachment from "../../becca/entities/battachment";import type BNote from "../../becca/entities/bnote";import noteService from "../../services/notes.js";import { convertOfficeToHtml } from "../../services/office_preview.js";+import { getPlatform } from "../../services/platform.js";import protected_session from "../../services/protected_session";import { downloadData, downloadNoteInt } from "../helpers";import { serveContentWithRanges } from "../partial_content";@@ -140,6 +141,9 @@ function isTrimmed(req: Request): boolean {return req.query.trim === "1";}+/** The most a `bytes=N-` request for media is answered with in standalone. */+const STANDALONE_OPEN_RANGE_LENGTH = 8 * 1024 * 1024;+function openPartialInt(noteOrAttachment: BNote | BAttachment, req: Request, res: Response) {if (noteOrAttachment.isProtected && !protected_session.isProtectedSessionAvailable()) {return res.status(401).send("Protected session not available");@@ -150,7 +154,10 @@ function openPartialInt(noteOrAttachment: BNote | BAttachment, req: Request, resfileName: noteOrAttachment.getFileName(),mimeType: noteOrAttachment.mime,// blobId is a content hash, so it is a stable ETag that changes only when the content does.- etag: noteOrAttachment.blobId+ etag: noteOrAttachment.blobId,+ // In standalone every response is buffered whole on the page and, in the mobile shells,+ // copied through a native bridge, so the rest of a large video cannot go out at once.+ maxOpenRangeLength: getPlatform().isStandalone ? STANDALONE_OPEN_RANGE_LENGTH : undefined});}diff --git a/packages/trilium-core/src/routes/partial_content.spec.ts b/packages/trilium-core/src/routes/partial_content.spec.tsindex 4c16833469..80d44abbcd 100644--- a/packages/trilium-core/src/routes/partial_content.spec.ts+++ b/packages/trilium-core/src/routes/partial_content.spec.ts@@ -13,7 +13,7 @@ interface ServeResult {}/** Serves `content` through the real handler, capturing what it wrote to the response. */-function serve(content: string | Uint8Array, range?: string, etag?: string): ServeResult {+function serve(content: string | Uint8Array, range?: string, etag?: string, maxOpenRangeLength?: number): ServeResult {const captured: ServeResult = { status: 200, headers: {}, body: "", byteLength: 0 };const res = {@@ -33,7 +33,7 @@ function serve(content: string | Uint8Array, range?: string, etag?: string): Ser} as unknown as Response;const req = { headers: range ? { range } : {} } as unknown as Request;- serveContentWithRanges(req, res, { content, fileName: "clip.mp3", mimeType: "audio/mpeg", etag });+ serveContentWithRanges(req, res, { content, fileName: "clip.mp3", mimeType: "audio/mpeg", etag, maxOpenRangeLength });return captured;}@@ -90,6 +90,24 @@ describe("serveContentWithRanges", () => {expect(serve(CONTENT, "bytes=-500").headers["Content-Range"]).toBe("bytes 0-9/10");});+ it("cuts an open-ended range to maxOpenRangeLength, and leaves every other form whole", () => {+ const capped = serve(CONTENT, "bytes=2-", undefined, 4);+ expect(capped.status).toBe(206);+ expect(capped.body).toBe("2345");+ expect(capped.headers["Content-Range"]).toBe("bytes 2-5/10");+ expect(capped.headers["Content-Length"]).toBe("4");++ // Near the end there is less left than the cap.+ expect(serve(CONTENT, "bytes=8-", undefined, 4).headers["Content-Range"]).toBe("bytes 8-9/10");++ // The client named these lengths itself, and a request with no range wants the whole body.+ expect(serve(CONTENT, "bytes=1-8", undefined, 4).body).toBe("12345678");+ expect(serve(CONTENT, "bytes=-6", undefined, 4).body).toBe("456789");+ const whole = serve(CONTENT, undefined, undefined, 4);+ expect(whole.status).toBe(200);+ expect(whole.body).toBe(CONTENT);+ });+it("clamps an end past the content instead of refusing it", () => {const res = serve(CONTENT, "bytes=8-999");diff --git a/packages/trilium-core/src/routes/partial_content.ts b/packages/trilium-core/src/routes/partial_content.tsindex 43a66ea331..a1ca1378ce 100644--- a/packages/trilium-core/src/routes/partial_content.ts+++ b/packages/trilium-core/src/routes/partial_content.ts@@ -10,6 +10,12 @@ export interface RangedContent {mimeType: string;/** A stable content identifier (a blobId is a content hash), sent as the ETag so a dropped stream can be resumed. */etag?: string;+ /**+ * The most bytes a `bytes=N-` range is answered with. A media element asks for the rest of+ * the file that way and then requests the next range itself, so a cap bounds every response+ * without breaking playback. Ranges with an explicit end are served as asked.+ */+ maxOpenRangeLength?: number;}/** A byte range with both ends inclusive — the form the HTTP `Range` header speaks in. */@@ -28,7 +34,7 @@ export interface ByteRange {* Express and the standalone browser router both implement, so one handler serves the server,* desktop and standalone (WASM) variants alike.*/-export function serveContentWithRanges(req: Request, res: Response, { content, fileName, mimeType, etag }: RangedContent) {+export function serveContentWithRanges(req: Request, res: Response, { content, fileName, mimeType, etag, maxOpenRangeLength }: RangedContent) {const bytes = wrapStringOrBuffer(content);const totalSize = bytes.byteLength;@@ -45,7 +51,7 @@ export function serveContentWithRanges(req: Request, res: Response, { content, flet range: ByteRange | null;try {- range = parseRangeHeader(rangeHeader, totalSize);+ range = parseRangeHeader(rangeHeader, totalSize, maxOpenRangeLength);} catch {res.setHeader("Content-Range", `bytes */${totalSize}`);return res.status(416).send(`Invalid value for Range: ${rangeHeader}`);@@ -74,7 +80,7 @@ const RANGE_PATTERN = /^bytes=(\d*)-(\d*)$/;* A header that cannot be parsed is ignored rather than rejected (RFC 9110 §14.2), so an odd client* still gets its content; only a well-formed range that falls outside the content is unsatisfiable.*/-function parseRangeHeader(rangeHeader: string | undefined, totalSize: number): ByteRange | null {+function parseRangeHeader(rangeHeader: string | undefined, totalSize: number, maxOpenRangeLength = Infinity): ByteRange | null {const match = rangeHeader ? RANGE_PATTERN.exec(rangeHeader.trim()) : null;if (!match || totalSize === 0) {return null;@@ -93,8 +99,12 @@ function parseRangeHeader(rangeHeader: string | undefined, totalSize: number): B}const start = Number(startValue);- // `bytes=500-` runs to the end, and an end past it is clamped rather than refused.- const end = endValue === "" ? totalSize - 1 : Math.min(Number(endValue), totalSize - 1);+ // `bytes=500-` runs to the end (or to `maxOpenRangeLength`), and an end past it is clamped+ // rather than refused.+ const end = Math.min(+ endValue === "" ? start + maxOpenRangeLength - 1 : Number(endValue),+ totalSize - 1+ );if (start > end || start >= totalSize) {throw new UnsatisfiableRangeError();