Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
build(flatpak): give the local build its own app ID
Elian Doran3add631a003c
diff --git a/.claude/skills/packaging-for-flathub/SKILL.md b/.claude/skills/packaging-for-flathub/SKILL.mdindex 9b15535825..ab820626cf 100644--- a/.claude/skills/packaging-for-flathub/SKILL.md+++ b/.claude/skills/packaging-for-flathub/SKILL.md@@ -62,23 +62,31 @@ pnpm desktop:flathub-build # scripts/flatpak/build-local.mtspnpm desktop:flathub-start```-`build-local.mts` renders the same manifest into `apps/desktop/out/flathub/` with the git-source swapped for a `type: dir` source over the working tree, so uncommitted changes are-built and nothing needs committing or pushing. `skip` matches exact relative paths, not-patterns, so the script lists every path `git ls-files --others --ignored --directory`-reports (~200: each `node_modules`, `dist`, …). `.git` stays in: the manifest's-`chore:update-build-info --from-commit` reads `git log`, and the local build keeps-Flathub's build commands unchanged. It regenerates `generated-sources.json` only when the-working tree's lockfile changes, and installs through `org.flatpak.Builder` as branch `local`, beside the Flathub install. An-install becomes flatpak's `current` branch, which the desktop launcher runs, so the script-puts the previous one back (`flatpak info` cannot report it once several are installed;-the script reads the `app/<id>/current` link).-`flathub:start` runs that branch on its own `trilium-data-local` and Electron data dirs-under `~/.var/app/org.triliumnotes.Trilium`, so a dev build never migrates the real-sandboxed database. Without `flatpak-node-generator` on PATH it runs the generator on the-Builder's Python from `~/.cache/flatpak-builder-tools` (cloned at the CI pin), raising-aiohttp 3.10's 8190-byte header limit, which GitHub's release responses exceed. A full-build takes about ten minutes.+`build-local.mts` renders the same manifest into `apps/desktop/out/flathub/` with two+changes, and installs it through `org.flatpak.Builder` for the user:++- **The git source becomes a `type: dir` source over the working tree**, so uncommitted+ changes are built and nothing needs committing or pushing. `skip` matches exact relative+ paths, not patterns, so the script lists every path `git ls-files -z --others --ignored+ --directory` reports (~200: each `node_modules`, `dist`, …). `.git` stays in: the+ manifest's `chore:update-build-info --from-commit` reads `git log`. That is also why a+ **linked worktree is refused** — its `.git` file points outside the sandbox.+- **The app ID becomes `org.triliumnotes.Trilium.Test`**, with its own launcher ("Trilium+ Notes (Test)"), `~/.var/app` data and portal autostart entry. An earlier version+ installed a `local` branch of the release's ID instead; that shadowed a system-wide+ release's launcher and let a dev run rewrite the release's autostart entry. The build+ commands install the desktop file, metainfo and icons under the release's name, and the+ manifest's `rename-desktop-file`/`rename-appdata-file`/`rename-icon` rename them and the+ IDs inside them.++`generated-sources.json` is regenerated only when the working tree's lockfile changes.+Without `flatpak-node-generator` on PATH, the generator runs on the Builder's Python from+`~/.cache/flatpak-builder-tools`, moved to the CI pin when it differs, raising aiohttp+3.10's 8190-byte header limit, which GitHub's release responses exceed. The script drops+`SOURCE_DATE_EPOCH` (a Nix shell's 1980 would date the export older than the installed+commit) and passes `--delete-build-dirs`: a failed build otherwise leaves a ~5 GB copy+of the repo under `.flatpak-builder/build/`, which `pnpm scripts:test` then also collects.+A full build takes about ten minutes.## Settled decisions and why (do not relitigate without new facts)diff --git a/apps/desktop/package.json b/apps/desktop/package.jsonindex 2da0b2514b..42a001fda4 100644--- a/apps/desktop/package.json+++ b/apps/desktop/package.json@@ -22,7 +22,7 @@"electron-forge:test-packaged": "pnpm electron-forge:package && tsx scripts/test-packaged.ts","electron-forge:start":"pnpm build && electron-forge start dist","flathub:build": "tsx ../../scripts/flatpak/build-local.mts",- "flathub:start": "flatpak run --branch=local --env=TRILIUM_DATA_DIR=$HOME/.var/app/org.triliumnotes.Trilium/data/trilium-data-local --env=TRILIUM_ELECTRON_DATA_DIR=$HOME/.var/app/org.triliumnotes.Trilium/config/trilium-electron-local org.triliumnotes.Trilium",+ "flathub:start": "flatpak run org.triliumnotes.Trilium.Test","e2e": "pnpm build && cross-env TRILIUM_INTEGRATION_TEST=memory-no-store TRILIUM_PORT=8082 TRILIUM_DATA_DIR=data-e2e TRILIUM_ELECTRON_DATA_DIR=data-e2e-electron-8082 ELECTRON_IS_DEV=0 playwright test","generate-setup-banners": "tsx electron-forge/setup-icon/generate/generate.mts","generate-dmg-background": "tsx electron-forge/dmg-background/generate.mts",diff --git a/scripts/flatpak/build-local.mts b/scripts/flatpak/build-local.mtsindex 23b9a8aed3..5a9ab2c20a 100644--- a/scripts/flatpak/build-local.mts+++ b/scripts/flatpak/build-local.mts@@ -1,19 +1,20 @@/*** Builds the Flathub flatpak from this checkout's working tree and installs it for the- * current user as the `local` branch of org.triliumnotes.Trilium, next to any install- * from Flathub.+ * current user as org.triliumnotes.Trilium.Test, next to any install of the release.*- * The manifest is the one ./scripts/flatpak/update-repo.mts writes for Flathub, except- * that its git source becomes a `dir` source over this checkout: the build sees the files- * on disk, uncommitted changes included, minus everything git ignores. `.git` stays in,- * because the manifest stamps the build info from `git log`.+ * The manifest is the one ./scripts/flatpak/update-repo.mts writes for Flathub, with two+ * changes. Its git source becomes a `dir` source over this checkout: the build sees the+ * files on disk, uncommitted changes included, minus everything git ignores. `.git` stays+ * in, because the manifest stamps the build info from `git log`. And the app ID gains a+ * `.Test` suffix, so the build has its own launcher, data directory and autostart entry+ * instead of taking over the release's.* `generated-sources.json` is regenerated only when the lockfile changes.** Usage:** pnpm exec tsx ./scripts/flatpak/build-local.mts*- * Then run it with `flatpak run --branch=local org.triliumnotes.Trilium`.+ * Then run it with `flatpak run org.triliumnotes.Trilium.Test`.** Builds through the org.flatpak.Builder flatpak. Without `flatpak-node-generator` on* PATH, the generator runs on that flatpak's Python from a flatpak-builder-tools@@ -22,9 +23,9 @@import { execFileSync } from "node:child_process";import { createHash } from "node:crypto";-import { existsSync, mkdirSync, readFileSync, readlinkSync, writeFileSync } from "node:fs";+import { existsSync, mkdirSync, readFileSync, statSync, writeFileSync } from "node:fs";import { homedir } from "node:os";-import { basename, join, resolve } from "node:path";+import { join, resolve } from "node:path";import { main as generateSources } from "./generate-sources.mjs";import { MANIFEST_NAME, renderManifest, writePackagingFiles } from "./update-repo.mjs";@@ -33,51 +34,42 @@ const REPO_ROOT = resolve(import.meta.dirname, "../..");const OUT_DIR = join(REPO_ROOT, "apps/desktop/out/flathub");const SOURCES_LOCKFILE_STAMP = join(OUT_DIR, "generated-sources.lockfile");const APP_ID = "org.triliumnotes.Trilium";-const LOCAL_BRANCH = "local";+const TEST_APP_ID = `${APP_ID}.Test`;// Must match the commit .github/workflows/release-flathub.yml installs.const GENERATOR_COMMIT = "41c20aa10819cdb2a4f3ca171758a96d1955c018";const GENERATOR_REPO = "https://github.com/flatpak/flatpak-builder-tools.git";export async function main() {+ checkNotWorktree();mkdirSync(OUT_DIR, { recursive: true });const { manifest } = await renderManifest("HEAD");- const ignored = parseIgnoredPaths(- git("ls-files", "--others", "--ignored", "--exclude-standard", "--directory")- );- writePackagingFiles(OUT_DIR, useLocalDirSource(manifest, REPO_ROOT, ignored));+ const ignored = parseIgnoredPaths(execFileSync("git", [+ "ls-files", "-z", "--others", "--ignored", "--exclude-standard", "--directory"+ ], { cwd: REPO_ROOT }).toString());+ writePackagingFiles(OUT_DIR, useTestAppId(useLocalDirSource(manifest, REPO_ROOT, ignored)));updateSources();- const currentBranch = getCurrentBranch();// A Nix shell sets SOURCE_DATE_EPOCH to 1980, which would date the export, and flatpak// refuses to install a commit older than the installed one.const { SOURCE_DATE_EPOCH: _, ...env } = process.env;execFileSync("flatpak", ["run", "org.flatpak.Builder",- "--user", "--install", "--force-clean",+ "--user", "--install", "--force-clean", "--delete-build-dirs","--install-deps-from=flathub",- `--default-branch=${LOCAL_BRANCH}`,"builddir", MANIFEST_NAME], { cwd: OUT_DIR, env, stdio: "inherit" });- // An install becomes the default branch, which the desktop launcher runs on the real data.- if (currentBranch && currentBranch !== LOCAL_BRANCH) {- execFileSync("flatpak", [ "make-current", "--user", APP_ID, currentBranch ], {- stdio: "inherit"- });- }- console.log(`Installed the working tree as the "${LOCAL_BRANCH}" branch.`);+ console.log(`Installed the working tree as ${TEST_APP_ID}.`);}/**- * The branch `flatpak run` picks without `--branch`, if the app is installed. `flatpak info`- * cannot tell: it refuses to pick one once several branches are installed.+ * In a linked worktree, `.git` is a file pointing into the main checkout, which the build+ * sandbox cannot reach, so the manifest's `git log` would fail halfway through the build.*/-function getCurrentBranch(): string | undefined {- const userDir = process.env.FLATPAK_USER_DIR ?? join(dataDir(), "flatpak");- try {- return basename(readlinkSync(join(userDir, "app", APP_ID, "current")));- } catch {- return undefined;+function checkNotWorktree() {+ if (statSync(join(REPO_ROOT, ".git")).isFile()) {+ throw new Error("This checkout is a linked git worktree, whose .git file points "+ + "outside the build sandbox. Build from the main checkout instead.");}}@@ -103,11 +95,32 @@ export function useLocalDirSource(manifest: string, repoDir: string, skip: strin].join("\n"));}-/** The `git ls-files --ignored --directory` output as paths, without those inside another. */+/**+ * Builds the app as TEST_APP_ID. The build commands install the desktop file, metainfo and+ * icons under the release's name, and flatpak-builder's `rename-*` options rename them and+ * the IDs inside them to the test app's.+ */+export function useTestAppId(manifest: string): string {+ const appIdLine = new RegExp(`^app-id: ${APP_ID.replaceAll(".", "\\.")}$`, "m");+ if (!appIdLine.test(manifest)) {+ throw new Error(`Found no "app-id: ${APP_ID}" line to rename.`);+ }+ return manifest+ .replace(appIdLine, [+ `app-id: ${TEST_APP_ID}`,+ `rename-desktop-file: ${APP_ID}.desktop`,+ `rename-appdata-file: ${APP_ID}.metainfo.xml`,+ `rename-icon: ${APP_ID}`,+ "desktop-file-name-suffix: \" (Test)\""+ ].join("\n"))+ .replaceAll("${FLATPAK_ID}", APP_ID);+}++/** NUL-separated `git ls-files -z --directory` output as paths, without those inside another. */export function parseIgnoredPaths(lsFilesOutput: string): string[] {- const paths = lsFilesOutput.split("\n")- .map((line) => line.trim().replace(/\/$/, ""))- .filter(Boolean);+ const paths = lsFilesOutput.split("\0")+ .filter(Boolean)+ .map((path) => path.replace(/\/$/, ""));return paths.filter((path) => !paths.some((other) => path.startsWith(`${other}/`)));}@@ -144,18 +157,30 @@ function generatorCommand(): string[] {// org.flatpak.Builder sees the host's home directory but not its /tmp.const cacheDir = process.env.XDG_CACHE_HOME || join(homedir(), ".cache");const checkout = join(cacheDir, "flatpak-builder-tools");- if (!existsSync(checkout)) {- execFileSync("git", [ "clone", "--quiet", GENERATOR_REPO, checkout ], { stdio: "inherit" });- execFileSync("git", [ "-C", checkout, "checkout", "--quiet", GENERATOR_COMMIT ], {- stdio: "inherit"- });- }+ checkOutGenerator(checkout);return ["flatpak", "run", "--command=python3", `--env=PYTHONPATH=${join(checkout, "node")}`,"org.flatpak.Builder", "-c", GENERATOR_BOOTSTRAP];}+/** Clones the generator on first use and moves an existing clone to GENERATOR_COMMIT. */+function checkOutGenerator(checkout: string) {+ if (!existsSync(checkout)) {+ execFileSync("git", [ "clone", "--quiet", GENERATOR_REPO, checkout ], { stdio: "inherit" });+ }+ const head = execFileSync("git", [ "-C", checkout, "rev-parse", "HEAD" ]).toString().trim();+ if (head === GENERATOR_COMMIT) {+ return;+ }+ execFileSync("git", [ "-C", checkout, "fetch", "--quiet", GENERATOR_REPO, GENERATOR_COMMIT ], {+ stdio: "inherit"+ });+ execFileSync("git", [ "-C", checkout, "checkout", "--quiet", "--detach", GENERATOR_COMMIT ], {+ stdio: "inherit"+ });+}+// The aiohttp in org.flatpak.Builder (3.10) rejects GitHub's release responses, whose// headers exceed its 8190-byte default, so the generator's sessions get larger limits.const GENERATOR_BOOTSTRAP = `\@@ -165,14 +190,6 @@ aiohttp.ClientSession.__init__ = functools.partialmethod(runpy.run_module("flatpak_node_generator", run_name="__main__", alter_sys=True)`;-function dataDir() {- return process.env.XDG_DATA_HOME || join(homedir(), ".local/share");-}--function git(...args: string[]): string {- return execFileSync("git", args, { cwd: REPO_ROOT }).toString().trim();-}-// Only when run as a script — the pure helpers above are imported by the spec.if (process.argv[1] === import.meta.filename) {main().catch((err) => {diff --git a/scripts/flatpak/build-local.spec.ts b/scripts/flatpak/build-local.spec.tsindex a769735b4f..cb88cdc0c2 100644--- a/scripts/flatpak/build-local.spec.ts+++ b/scripts/flatpak/build-local.spec.ts@@ -1,6 +1,6 @@import { describe, expect, it } from "vitest";-import { parseIgnoredPaths, useLocalDirSource } from "./build-local.mjs";+import { parseIgnoredPaths, useLocalDirSource, useTestAppId } from "./build-local.mjs";const SOURCES = (pin: string) => `\sources:@@ -38,12 +38,45 @@ describe("useLocalDirSource", () => {});});+describe("useTestAppId", () => {+ it("renames the app and builds its exports from the release's files", () => {+ const manifest = `\+app-id: org.triliumnotes.Trilium+runtime: org.freedesktop.Platform+modules:+ - name: trilium+ build-commands:+ - install -Dm644 apps/desktop/flatpak/\${FLATPAK_ID}.desktop /app/share/applications/\${FLATPAK_ID}.desktop+ - install -Dm644 apps/desktop/flatpak/\${FLATPAK_ID}.metainfo.xml /app/share/metainfo/\${FLATPAK_ID}.metainfo.xml+`;+ expect(useTestAppId(manifest)).toBe(`\+app-id: org.triliumnotes.Trilium.Test+rename-desktop-file: org.triliumnotes.Trilium.desktop+rename-appdata-file: org.triliumnotes.Trilium.metainfo.xml+rename-icon: org.triliumnotes.Trilium+desktop-file-name-suffix: " (Test)"+runtime: org.freedesktop.Platform+modules:+ - name: trilium+ build-commands:+ - install -Dm644 apps/desktop/flatpak/org.triliumnotes.Trilium.desktop /app/share/applications/org.triliumnotes.Trilium.desktop+ - install -Dm644 apps/desktop/flatpak/org.triliumnotes.Trilium.metainfo.xml /app/share/metainfo/org.triliumnotes.Trilium.metainfo.xml+`);+ });++ it("fails on a manifest for another app", () => {+ expect(() => useTestAppId("app-id: org.example.App\n")).toThrow(/app-id/);+ });+});+describe("parseIgnoredPaths", () => {- it("strips directory slashes and drops paths inside another ignored one", () => {- const output = ".flatpak-builder/\n.flatpak-builder/ccache/\napps/client/dist/\n"- + "apps/client/tsconfig.tsbuildinfo\nnode_modules/\nnode_modules_backup/\n";+ it("keeps each NUL-separated path exactly, minus directory slashes and nested paths", () => {+ const output = [+ ".flatpak-builder/", ".flatpak-builder/ccache/", "apps/client/dist/",+ " notes with spaces .md", "Ünïcode \"quoted\"/", "node_modules/", "node_modules_backup/", ""+ ].join("\0");expect(parseIgnoredPaths(output)).toEqual([- ".flatpak-builder", "apps/client/dist", "apps/client/tsconfig.tsbuildinfo",+ ".flatpak-builder", "apps/client/dist", " notes with spaces .md", "Ünïcode \"quoted\"","node_modules", "node_modules_backup"]);});