Skip to content

Build your personal knowledge base with Trilium Notes

build(flatpak): give the local build its own app ID

Elian Doran3add631a003c
diff --git a/.claude/skills/packaging-for-flathub/SKILL.md b/.claude/skills/packaging-for-flathub/SKILL.md
index 9b15535825..ab820626cf 100644
--- a/.claude/skills/packaging-for-flathub/SKILL.md
+++ b/.claude/skills/packaging-for-flathub/SKILL.md
@@ -62,23 +62,31 @@ pnpm desktop:flathub-build # scripts/flatpak/build-local.mts
pnpm desktop:flathub-start
```
-`build-local.mts` renders the same manifest into `apps/desktop/out/flathub/` with the git
-source swapped for a `type: dir` source over the working tree, so uncommitted changes are
-built and nothing needs committing or pushing. `skip` matches exact relative paths, not
-patterns, so the script lists every path `git ls-files --others --ignored --directory`
-reports (~200: each `node_modules`, `dist`, …). `.git` stays in: the manifest's
-`chore:update-build-info --from-commit` reads `git log`, and the local build keeps
-Flathub's build commands unchanged. It regenerates `generated-sources.json` only when the
-working tree's lockfile changes, and installs through `org.flatpak.Builder` as branch `local`, beside the Flathub install. An
-install becomes flatpak's `current` branch, which the desktop launcher runs, so the script
-puts the previous one back (`flatpak info` cannot report it once several are installed;
-the script reads the `app/<id>/current` link).
-`flathub:start` runs that branch on its own `trilium-data-local` and Electron data dirs
-under `~/.var/app/org.triliumnotes.Trilium`, so a dev build never migrates the real
-sandboxed database. Without `flatpak-node-generator` on PATH it runs the generator on the
-Builder's Python from `~/.cache/flatpak-builder-tools` (cloned at the CI pin), raising
-aiohttp 3.10's 8190-byte header limit, which GitHub's release responses exceed. A full
-build takes about ten minutes.
+`build-local.mts` renders the same manifest into `apps/desktop/out/flathub/` with two
+changes, and installs it through `org.flatpak.Builder` for the user:
+
+- **The git source becomes a `type: dir` source over the working tree**, so uncommitted
+ changes are built and nothing needs committing or pushing. `skip` matches exact relative
+ paths, not patterns, so the script lists every path `git ls-files -z --others --ignored
+ --directory` reports (~200: each `node_modules`, `dist`, …). `.git` stays in: the
+ manifest's `chore:update-build-info --from-commit` reads `git log`. That is also why a
+ **linked worktree is refused** — its `.git` file points outside the sandbox.
+- **The app ID becomes `org.triliumnotes.Trilium.Test`**, with its own launcher ("Trilium
+ Notes (Test)"), `~/.var/app` data and portal autostart entry. An earlier version
+ installed a `local` branch of the release's ID instead; that shadowed a system-wide
+ release's launcher and let a dev run rewrite the release's autostart entry. The build
+ commands install the desktop file, metainfo and icons under the release's name, and the
+ manifest's `rename-desktop-file`/`rename-appdata-file`/`rename-icon` rename them and the
+ IDs inside them.
+
+`generated-sources.json` is regenerated only when the working tree's lockfile changes.
+Without `flatpak-node-generator` on PATH, the generator runs on the Builder's Python from
+`~/.cache/flatpak-builder-tools`, moved to the CI pin when it differs, raising aiohttp
+3.10's 8190-byte header limit, which GitHub's release responses exceed. The script drops
+`SOURCE_DATE_EPOCH` (a Nix shell's 1980 would date the export older than the installed
+commit) and passes `--delete-build-dirs`: a failed build otherwise leaves a ~5 GB copy
+of the repo under `.flatpak-builder/build/`, which `pnpm scripts:test` then also collects.
+A full build takes about ten minutes.
## Settled decisions and why (do not relitigate without new facts)
diff --git a/apps/desktop/package.json b/apps/desktop/package.json
index 2da0b2514b..42a001fda4 100644
--- a/apps/desktop/package.json
+++ b/apps/desktop/package.json
@@ -22,7 +22,7 @@
"electron-forge:test-packaged": "pnpm electron-forge:package && tsx scripts/test-packaged.ts",
"electron-forge:start":"pnpm build && electron-forge start dist",
"flathub:build": "tsx ../../scripts/flatpak/build-local.mts",
- "flathub:start": "flatpak run --branch=local --env=TRILIUM_DATA_DIR=$HOME/.var/app/org.triliumnotes.Trilium/data/trilium-data-local --env=TRILIUM_ELECTRON_DATA_DIR=$HOME/.var/app/org.triliumnotes.Trilium/config/trilium-electron-local org.triliumnotes.Trilium",
+ "flathub:start": "flatpak run org.triliumnotes.Trilium.Test",
"e2e": "pnpm build && cross-env TRILIUM_INTEGRATION_TEST=memory-no-store TRILIUM_PORT=8082 TRILIUM_DATA_DIR=data-e2e TRILIUM_ELECTRON_DATA_DIR=data-e2e-electron-8082 ELECTRON_IS_DEV=0 playwright test",
"generate-setup-banners": "tsx electron-forge/setup-icon/generate/generate.mts",
"generate-dmg-background": "tsx electron-forge/dmg-background/generate.mts",
diff --git a/scripts/flatpak/build-local.mts b/scripts/flatpak/build-local.mts
index 23b9a8aed3..5a9ab2c20a 100644
--- a/scripts/flatpak/build-local.mts
+++ b/scripts/flatpak/build-local.mts
@@ -1,19 +1,20 @@
/**
* Builds the Flathub flatpak from this checkout's working tree and installs it for the
- * current user as the `local` branch of org.triliumnotes.Trilium, next to any install
- * from Flathub.
+ * current user as org.triliumnotes.Trilium.Test, next to any install of the release.
*
- * The manifest is the one ./scripts/flatpak/update-repo.mts writes for Flathub, except
- * that its git source becomes a `dir` source over this checkout: the build sees the files
- * on disk, uncommitted changes included, minus everything git ignores. `.git` stays in,
- * because the manifest stamps the build info from `git log`.
+ * The manifest is the one ./scripts/flatpak/update-repo.mts writes for Flathub, with two
+ * changes. Its git source becomes a `dir` source over this checkout: the build sees the
+ * files on disk, uncommitted changes included, minus everything git ignores. `.git` stays
+ * in, because the manifest stamps the build info from `git log`. And the app ID gains a
+ * `.Test` suffix, so the build has its own launcher, data directory and autostart entry
+ * instead of taking over the release's.
* `generated-sources.json` is regenerated only when the lockfile changes.
*
* Usage:
*
* pnpm exec tsx ./scripts/flatpak/build-local.mts
*
- * Then run it with `flatpak run --branch=local org.triliumnotes.Trilium`.
+ * Then run it with `flatpak run org.triliumnotes.Trilium.Test`.
*
* Builds through the org.flatpak.Builder flatpak. Without `flatpak-node-generator` on
* PATH, the generator runs on that flatpak's Python from a flatpak-builder-tools
@@ -22,9 +23,9 @@
import { execFileSync } from "node:child_process";
import { createHash } from "node:crypto";
-import { existsSync, mkdirSync, readFileSync, readlinkSync, writeFileSync } from "node:fs";
+import { existsSync, mkdirSync, readFileSync, statSync, writeFileSync } from "node:fs";
import { homedir } from "node:os";
-import { basename, join, resolve } from "node:path";
+import { join, resolve } from "node:path";
import { main as generateSources } from "./generate-sources.mjs";
import { MANIFEST_NAME, renderManifest, writePackagingFiles } from "./update-repo.mjs";
@@ -33,51 +34,42 @@ const REPO_ROOT = resolve(import.meta.dirname, "../..");
const OUT_DIR = join(REPO_ROOT, "apps/desktop/out/flathub");
const SOURCES_LOCKFILE_STAMP = join(OUT_DIR, "generated-sources.lockfile");
const APP_ID = "org.triliumnotes.Trilium";
-const LOCAL_BRANCH = "local";
+const TEST_APP_ID = `${APP_ID}.Test`;
// Must match the commit .github/workflows/release-flathub.yml installs.
const GENERATOR_COMMIT = "41c20aa10819cdb2a4f3ca171758a96d1955c018";
const GENERATOR_REPO = "https://github.com/flatpak/flatpak-builder-tools.git";
export async function main() {
+ checkNotWorktree();
mkdirSync(OUT_DIR, { recursive: true });
const { manifest } = await renderManifest("HEAD");
- const ignored = parseIgnoredPaths(
- git("ls-files", "--others", "--ignored", "--exclude-standard", "--directory")
- );
- writePackagingFiles(OUT_DIR, useLocalDirSource(manifest, REPO_ROOT, ignored));
+ const ignored = parseIgnoredPaths(execFileSync("git", [
+ "ls-files", "-z", "--others", "--ignored", "--exclude-standard", "--directory"
+ ], { cwd: REPO_ROOT }).toString());
+ writePackagingFiles(OUT_DIR, useTestAppId(useLocalDirSource(manifest, REPO_ROOT, ignored)));
updateSources();
- const currentBranch = getCurrentBranch();
// A Nix shell sets SOURCE_DATE_EPOCH to 1980, which would date the export, and flatpak
// refuses to install a commit older than the installed one.
const { SOURCE_DATE_EPOCH: _, ...env } = process.env;
execFileSync("flatpak", [
"run", "org.flatpak.Builder",
- "--user", "--install", "--force-clean",
+ "--user", "--install", "--force-clean", "--delete-build-dirs",
"--install-deps-from=flathub",
- `--default-branch=${LOCAL_BRANCH}`,
"builddir", MANIFEST_NAME
], { cwd: OUT_DIR, env, stdio: "inherit" });
- // An install becomes the default branch, which the desktop launcher runs on the real data.
- if (currentBranch && currentBranch !== LOCAL_BRANCH) {
- execFileSync("flatpak", [ "make-current", "--user", APP_ID, currentBranch ], {
- stdio: "inherit"
- });
- }
- console.log(`Installed the working tree as the "${LOCAL_BRANCH}" branch.`);
+ console.log(`Installed the working tree as ${TEST_APP_ID}.`);
}
/**
- * The branch `flatpak run` picks without `--branch`, if the app is installed. `flatpak info`
- * cannot tell: it refuses to pick one once several branches are installed.
+ * In a linked worktree, `.git` is a file pointing into the main checkout, which the build
+ * sandbox cannot reach, so the manifest's `git log` would fail halfway through the build.
*/
-function getCurrentBranch(): string | undefined {
- const userDir = process.env.FLATPAK_USER_DIR ?? join(dataDir(), "flatpak");
- try {
- return basename(readlinkSync(join(userDir, "app", APP_ID, "current")));
- } catch {
- return undefined;
+function checkNotWorktree() {
+ if (statSync(join(REPO_ROOT, ".git")).isFile()) {
+ throw new Error("This checkout is a linked git worktree, whose .git file points "
+ + "outside the build sandbox. Build from the main checkout instead.");
}
}
@@ -103,11 +95,32 @@ export function useLocalDirSource(manifest: string, repoDir: string, skip: strin
].join("\n"));
}
-/** The `git ls-files --ignored --directory` output as paths, without those inside another. */
+/**
+ * Builds the app as TEST_APP_ID. The build commands install the desktop file, metainfo and
+ * icons under the release's name, and flatpak-builder's `rename-*` options rename them and
+ * the IDs inside them to the test app's.
+ */
+export function useTestAppId(manifest: string): string {
+ const appIdLine = new RegExp(`^app-id: ${APP_ID.replaceAll(".", "\\.")}$`, "m");
+ if (!appIdLine.test(manifest)) {
+ throw new Error(`Found no "app-id: ${APP_ID}" line to rename.`);
+ }
+ return manifest
+ .replace(appIdLine, [
+ `app-id: ${TEST_APP_ID}`,
+ `rename-desktop-file: ${APP_ID}.desktop`,
+ `rename-appdata-file: ${APP_ID}.metainfo.xml`,
+ `rename-icon: ${APP_ID}`,
+ "desktop-file-name-suffix: \" (Test)\""
+ ].join("\n"))
+ .replaceAll("${FLATPAK_ID}", APP_ID);
+}
+
+/** NUL-separated `git ls-files -z --directory` output as paths, without those inside another. */
export function parseIgnoredPaths(lsFilesOutput: string): string[] {
- const paths = lsFilesOutput.split("\n")
- .map((line) => line.trim().replace(/\/$/, ""))
- .filter(Boolean);
+ const paths = lsFilesOutput.split("\0")
+ .filter(Boolean)
+ .map((path) => path.replace(/\/$/, ""));
return paths.filter((path) => !paths.some((other) => path.startsWith(`${other}/`)));
}
@@ -144,18 +157,30 @@ function generatorCommand(): string[] {
// org.flatpak.Builder sees the host's home directory but not its /tmp.
const cacheDir = process.env.XDG_CACHE_HOME || join(homedir(), ".cache");
const checkout = join(cacheDir, "flatpak-builder-tools");
- if (!existsSync(checkout)) {
- execFileSync("git", [ "clone", "--quiet", GENERATOR_REPO, checkout ], { stdio: "inherit" });
- execFileSync("git", [ "-C", checkout, "checkout", "--quiet", GENERATOR_COMMIT ], {
- stdio: "inherit"
- });
- }
+ checkOutGenerator(checkout);
return [
"flatpak", "run", "--command=python3", `--env=PYTHONPATH=${join(checkout, "node")}`,
"org.flatpak.Builder", "-c", GENERATOR_BOOTSTRAP
];
}
+/** Clones the generator on first use and moves an existing clone to GENERATOR_COMMIT. */
+function checkOutGenerator(checkout: string) {
+ if (!existsSync(checkout)) {
+ execFileSync("git", [ "clone", "--quiet", GENERATOR_REPO, checkout ], { stdio: "inherit" });
+ }
+ const head = execFileSync("git", [ "-C", checkout, "rev-parse", "HEAD" ]).toString().trim();
+ if (head === GENERATOR_COMMIT) {
+ return;
+ }
+ execFileSync("git", [ "-C", checkout, "fetch", "--quiet", GENERATOR_REPO, GENERATOR_COMMIT ], {
+ stdio: "inherit"
+ });
+ execFileSync("git", [ "-C", checkout, "checkout", "--quiet", "--detach", GENERATOR_COMMIT ], {
+ stdio: "inherit"
+ });
+}
+
// The aiohttp in org.flatpak.Builder (3.10) rejects GitHub's release responses, whose
// headers exceed its 8190-byte default, so the generator's sessions get larger limits.
const GENERATOR_BOOTSTRAP = `\
@@ -165,14 +190,6 @@ aiohttp.ClientSession.__init__ = functools.partialmethod(
runpy.run_module("flatpak_node_generator", run_name="__main__", alter_sys=True)
`;
-function dataDir() {
- return process.env.XDG_DATA_HOME || join(homedir(), ".local/share");
-}
-
-function git(...args: string[]): string {
- return execFileSync("git", args, { cwd: REPO_ROOT }).toString().trim();
-}
-
// Only when run as a script — the pure helpers above are imported by the spec.
if (process.argv[1] === import.meta.filename) {
main().catch((err) => {
diff --git a/scripts/flatpak/build-local.spec.ts b/scripts/flatpak/build-local.spec.ts
index a769735b4f..cb88cdc0c2 100644
--- a/scripts/flatpak/build-local.spec.ts
+++ b/scripts/flatpak/build-local.spec.ts
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
-import { parseIgnoredPaths, useLocalDirSource } from "./build-local.mjs";
+import { parseIgnoredPaths, useLocalDirSource, useTestAppId } from "./build-local.mjs";
const SOURCES = (pin: string) => `\
sources:
@@ -38,12 +38,45 @@ describe("useLocalDirSource", () => {
});
});
+describe("useTestAppId", () => {
+ it("renames the app and builds its exports from the release's files", () => {
+ const manifest = `\
+app-id: org.triliumnotes.Trilium
+runtime: org.freedesktop.Platform
+modules:
+ - name: trilium
+ build-commands:
+ - install -Dm644 apps/desktop/flatpak/\${FLATPAK_ID}.desktop /app/share/applications/\${FLATPAK_ID}.desktop
+ - install -Dm644 apps/desktop/flatpak/\${FLATPAK_ID}.metainfo.xml /app/share/metainfo/\${FLATPAK_ID}.metainfo.xml
+`;
+ expect(useTestAppId(manifest)).toBe(`\
+app-id: org.triliumnotes.Trilium.Test
+rename-desktop-file: org.triliumnotes.Trilium.desktop
+rename-appdata-file: org.triliumnotes.Trilium.metainfo.xml
+rename-icon: org.triliumnotes.Trilium
+desktop-file-name-suffix: " (Test)"
+runtime: org.freedesktop.Platform
+modules:
+ - name: trilium
+ build-commands:
+ - install -Dm644 apps/desktop/flatpak/org.triliumnotes.Trilium.desktop /app/share/applications/org.triliumnotes.Trilium.desktop
+ - install -Dm644 apps/desktop/flatpak/org.triliumnotes.Trilium.metainfo.xml /app/share/metainfo/org.triliumnotes.Trilium.metainfo.xml
+`);
+ });
+
+ it("fails on a manifest for another app", () => {
+ expect(() => useTestAppId("app-id: org.example.App\n")).toThrow(/app-id/);
+ });
+});
+
describe("parseIgnoredPaths", () => {
- it("strips directory slashes and drops paths inside another ignored one", () => {
- const output = ".flatpak-builder/\n.flatpak-builder/ccache/\napps/client/dist/\n"
- + "apps/client/tsconfig.tsbuildinfo\nnode_modules/\nnode_modules_backup/\n";
+ it("keeps each NUL-separated path exactly, minus directory slashes and nested paths", () => {
+ const output = [
+ ".flatpak-builder/", ".flatpak-builder/ccache/", "apps/client/dist/",
+ " notes with spaces .md", "Ünïcode \"quoted\"/", "node_modules/", "node_modules_backup/", ""
+ ].join("\0");
expect(parseIgnoredPaths(output)).toEqual([
- ".flatpak-builder", "apps/client/dist", "apps/client/tsconfig.tsbuildinfo",
+ ".flatpak-builder", "apps/client/dist", " notes with spaces .md", "Ünïcode \"quoted\"",
"node_modules", "node_modules_backup"
]);
});