Skip to content

Build your personal knowledge base with Trilium Notes

feat(server): color log output in the terminal

Elian Doran1cd26ae4b8be
diff --git a/apps/server/src/assets/help/help_content.json b/apps/server/src/assets/help/help_content.json
index 6ca000b257..deb3570991 100644
--- a/apps/server/src/assets/help/help_content.json
+++ b/apps/server/src/assets/help/help_content.json
@@ -188,7 +188,7 @@
"_help_wy8So3yZZlH9": "<p>Go to <a href=\"https://github.com/TriliumNext/Trilium/issues\">Issues · TriliumNext/Trilium</a> and press “New issue”.</p><p>If you are reporting a bug, select “Bug Report” and fill in the details.</p>",
"_help_x59R8J8KV5Bp": "<p><img src=\"{{helpAssets}}/User%20Guide/Troubleshooting/Anonymized%20Database_image.jpg\"></p><p>In certain scenarios, understanding the structure of a database is crucial for troubleshooting issues. However, sharing your actual <a href=\"#root/_help_wX4HbRucYSDD\">database</a> file with personal notes is not advisable. To address this, Trilium offers a feature to anonymize the database. This feature can be accessed via Menu -&gt; Options -&gt; Advanced tab.</p><p>This feature creates a copy of your database with all sensitive data removed. Specifically, it strips out note titles, contents, revisions, history, and some non-system attributes while retaining the overall structure and metadata, such as modification dates. After anonymization, the database undergoes a <a href=\"https://sqlite.org/lang_vacuum.html\">vacuuming process</a> to ensure no sensitive data remnants remain in the file. The anonymized database is saved in the <code>anonymized</code> directory within the <a href=\"#root/_help_tAassRL4RSQL\">data directory</a>, making it safe to share with bug reports.</p><p>This will create a copy of your document and remove all sensitive data (currently note titles, contents, revisions, history and some of the options, and non-system attributes) while leaving all structure and metadata (e.g. date of last change). After this is done, the database is <a href=\"https://sqlite.org/lang_vacuum.html\">VACUUMed</a> to make sure there's no stale sensitive data in the document file. The resulting file is stored in <code>anonymized</code> directory (placed in the <a href=\"#root/_help_tAassRL4RSQL\">data directory</a>). You can safely attach it to a bug report.</p><h2>Command Line Anonymization</h2><p>If your <a href=\"#root/_help_wX4HbRucYSDD\">database</a> is corrupted to the point where Trilium cannot start, the anonymization process can still be executed via the command line:</p><pre><code class=\"language-text-x-trilium-auto\">node src/anonymize.js</code></pre><p>Run this command from the directory containing Trilium's source files, typically found in the <code>resources/app</code> directory for desktop builds.</p>",
"_help_qzNzp9LYQyPT": "<p>It's important to provide all available error logs together with bug reports.</p><p>There are two types of error logs, both of which are useful when reporting bugs.</p><ul><li><a class=\"reference-link\" href=\"#root/_help_bnyigUA2UK7s\">Backend (server) logs</a></li><li><a class=\"reference-link\" href=\"#root/_help_9yEHzMyFirZR\">Frontend logs</a></li></ul><h2>Providing sensitive data</h2><p>If you don't feel comfortable attaching the logs or anything sensitive to the public GitHub issues, feel free to contact the devs in our Matrix <a href=\"https://github.com/TriliumNext/Trilium#-discuss-with-us\">support channel</a>.</p><p>Use this email to also provide anything which could assist in analysing the bug - e.g. files/images/ZIPs being imported or <a href=\"#root/_help_x59R8J8KV5Bp\">anonymized database</a>.</p><h3>Exporting note subtree for reproduction</h3><p>Often times, bugs manifest themselves in specific notes and having them would greatly ease reproduction and fixing.</p><p>In such case, please export the relevant note subtree by right-clicking it on the left tree, choosing Export - HTML as ZIP:</p><p><img src=\"{{helpAssets}}/User%20Guide/Troubleshooting/Error%20logs_error-logs-export-subtree.png\"></p>",
- "_help_bnyigUA2UK7s": "<h2>Accessing via the backend log</h2><p>In the&nbsp;<a class=\"reference-link\" href=\"#root/_help_x3i7MxGccDuM\">Global menu</a>, go to <em>Advanced</em> → <em>Show backend log</em>. This will display the current backend log (i.e. today's), with the historical information available only on disk (see below).</p><p>Interaction since v0.104.0:</p><ul><li>The file can be downloaded as a text file using the&nbsp;<span class=\"tn-icon bx bx-download\"></span> button in the&nbsp;<a class=\"reference-link\" href=\"#root/_help_8YBEPzcpUgxw\">Note menu</a>&nbsp;area.</li><li>The file can also be searched just like a normal&nbsp;<a class=\"reference-link\" href=\"#root/_help_6f9hih2hXXZk\">Code</a>&nbsp;note.</li></ul><h2>Location on the disk</h2><p>Backend logs are stored on the file system. To find them, open the&nbsp;<a class=\"reference-link\" href=\"#root/_help_tAassRL4RSQL\">Data directory</a>, go to the <code>log</code> subdirectory and find the latest log file, e.g. <code>trilium-2022-12-14.log</code>.&nbsp;</p><h2>Reporting backend bugs</h2><p>You can attach the whole file to the bug report (preferable) or open it and copy-paste only the last lines / lines you believe are relevant.</p><h2>Customizing the retention of logs</h2><p>The backend logs are fully managed by the Trilium server. By default the last 90 days worth of logs are kept; the logs older than that are deleted in order to reduce the space consumption.</p><p>It's possible to change the retention period by modifying the&nbsp;<a class=\"reference-link\" href=\"#root/_help_Gzjqa934BdH4\">Configuration (config.ini or environment variables)</a>&nbsp;via the <code>.ini</code> file:</p><pre><code class=\"language-text-x-trilium-auto\">[Logging]\nretentionDays=7</code></pre><p>Or via the environment variable <code>TRILIUM_LOGGING_RETENTION_DAYS</code>.</p><p>Special cases:</p><ul><li>Positive values indicate the number of days worth of logs to keep</li><li>A value of 0 results with the default value (90 days) to be used</li><li>Negative values (e.g. <code>-1</code>) result with all logs to be kept, irrespective how ancient and numerous (and</li></ul><aside class=\"admonition note\"><p>If you set the retention days to a low number, you might notice that not all the log files are being deleted. This is because a minimum number of logs (7 at the time of writing) is maintained at all times.</p></aside>",
+ "_help_bnyigUA2UK7s": "<h2>Accessing via the backend log</h2><p>In the&nbsp;<a class=\"reference-link\" href=\"#root/_help_x3i7MxGccDuM\">Global menu</a>, go to <em>Advanced</em> → <em>Show backend log</em>. This will display the current backend log (i.e. today's), with the historical information available only on disk (see below).</p><p>Interaction since v0.104.0:</p><ul><li>The file can be downloaded as a text file using the&nbsp;<span class=\"tn-icon bx bx-download\"></span> button in the&nbsp;<a class=\"reference-link\" href=\"#root/_help_8YBEPzcpUgxw\">Note menu</a>&nbsp;area.</li><li>The file can also be searched just like a normal&nbsp;<a class=\"reference-link\" href=\"#root/_help_6f9hih2hXXZk\">Code</a>&nbsp;note.</li></ul><h2>Location on the disk</h2><p>Backend logs are stored on the file system. To find them, open the&nbsp;<a class=\"reference-link\" href=\"#root/_help_tAassRL4RSQL\">Data directory</a>, go to the <code>log</code> subdirectory and find the latest log file, e.g. <code>trilium-2022-12-14.log</code>.&nbsp;</p><h2>In the terminal</h2><p>The server also prints every log entry to its standard output, which is what <code>docker logs</code> or <code>journalctl</code> show. When that output is a terminal, each entry starts with its time and is colored the same way as the backend log in the app: request status codes by class, slow SQL queries, and errors in red. Output to a file, a pipe or a container log stays plain text without the time, since those tools add their own.</p><p>To turn the colors off in a terminal, set the <code>NO_COLOR</code> environment variable; to force them on elsewhere, set <code>FORCE_COLOR=1</code>.</p><h2>Reporting backend bugs</h2><p>You can attach the whole file to the bug report (preferable) or open it and copy-paste only the last lines / lines you believe are relevant.</p><h2>Customizing the retention of logs</h2><p>The backend logs are fully managed by the Trilium server. By default the last 90 days worth of logs are kept; the logs older than that are deleted in order to reduce the space consumption.</p><p>It's possible to change the retention period by modifying the&nbsp;<a class=\"reference-link\" href=\"#root/_help_Gzjqa934BdH4\">Configuration (config.ini or environment variables)</a>&nbsp;via the <code>.ini</code> file:</p><pre><code class=\"language-text-x-trilium-auto\">[Logging]\nretentionDays=7</code></pre><p>Or via the environment variable <code>TRILIUM_LOGGING_RETENTION_DAYS</code>.</p><p>Special cases:</p><ul><li>Positive values indicate the number of days worth of logs to keep</li><li>A value of 0 results with the default value (90 days) to be used</li><li>Negative values (e.g. <code>-1</code>) result with all logs to be kept, irrespective how ancient and numerous (and</li></ul><aside class=\"admonition note\"><p>If you set the retention days to a low number, you might notice that not all the log files are being deleted. This is because a minimum number of logs (7 at the time of writing) is maintained at all times.</p></aside>",
"_help_9yEHzMyFirZR": "<p>To provide frontend logs, we need to open the Developer Console. Often the easiest way is to press <kbd>Ctrl</kbd>-<kbd>Shift</kbd>-<kbd>I</kbd> which should work in most browsers (and desktop app). Make sure that the error producing action happened right before you copy&amp;paste the errors, the console is cleared on app restart.</p><p>If that doesn't work, then:</p><ul><li>in Trilium desktop app, go to top-left menu button -&gt; Advanced -&gt; Open dev tools</li><li>In Firefox/Chrome right-click anywhere in the page and click Inspect:</li></ul><p><img src=\"{{helpAssets}}/User%20Guide/Troubleshooting/Error%20logs/Frontend%20logs_error-logs-firefox-context-men.png\"></p><p>Once you have Dev Tools open, click on \"Console\" tab:</p><p><img src=\"{{helpAssets}}/User%20Guide/Troubleshooting/Error%20logs/Frontend%20logs_image.jpg\"></p><p>Copy-paste (or screenshot) the logs. It's better to provide not just errors, but the whole log, which might provide context while analyzing the bug.</p>",
"_help_vdlYGAcpXAgc": "<p>Synchronization can sometimes take a long amount of time in order to compute the items that require update. When running behind a reverse proxy, the request can time out.</p><p>The solution is to increase the timeout at proxy level.</p><h2>Nginx</h2><p>Add the following to the configuration file:</p><pre><code class=\"language-text-x-nginx-conf\">proxy_connect_timeout 300;\nproxy_send_timeout 300;\nproxy_read_timeout 300;\nsend_timeout 300;</code></pre><p>And restart the server.</p><p>See&nbsp;<a href=\"#root/_help_ud6MShXL4WpO\">Nginx Proxy Setup</a>&nbsp;for more information about the Nginx setup.</p><p>If it still doesn't work, try increasing the timeout.</p>",
"_help_s8alTXmpFR61": "<p>Some changes to the application will not take effect immediately and as such it might require to manually reload or refresh the application.</p><p>To do so, simply press <kbd>Ctrl</kbd>+<kbd>Shift</kbd>+<kbd>R</kbd>.</p>",
diff --git a/apps/server/src/log_provider.spec.ts b/apps/server/src/log_provider.spec.ts
index 7dcf41997b..627fd27b2f 100644
--- a/apps/server/src/log_provider.spec.ts
+++ b/apps/server/src/log_provider.spec.ts
@@ -1,7 +1,48 @@
import type { Request, Response } from "express";
+import { styleText } from "node:util";
import { describe, expect, it, vi } from "vitest";
-import ServerLogService from "./log_provider.js";
+import ServerLogService, { colorizeLogEntry } from "./log_provider.js";
+
+describe("colorizeLogEntry", () => {
+ const ts = paint("gray", "12:34:56.789");
+
+ it("colours HTTP request lines like the Backend Log note", () => {
+ expect(colorizeLogEntry("Slow 404 GET /api/notes/abc with 15 bytes took 12ms", "12:34:56.789")).toBe(
+ `${ts} ${paint([ "bold", "yellow" ], "Slow")} ${paint([ "bold", "yellow" ], "404")} `
+ + `${paint([ "bold", "blueBright" ], "GET")} ${paint("cyan", "/api/notes/abc")} with 15 bytes took 12ms`
+ );
+
+ const statuses = { 200: "green", 304: "gray", 500: "red" } as const;
+ for (const [ status, color ] of Object.entries(statuses)) {
+ expect(colorizeLogEntry(`${status} POST /api/x with 1 bytes took 1ms`, "12:34:56.789")).toBe(
+ `${ts} ${paint([ "bold", color ], status)} ${paint([ "bold", "blueBright" ], "POST")} `
+ + `${paint("cyan", "/api/x")} with 1 bytes took 1ms`
+ );
+ }
+ });
+
+ it("colours slow SQL queries, with and without the statement", () => {
+ expect(colorizeLogEntry("Slow query took 78ms: SELECT * FROM notes", "12:34:56.789")).toBe(
+ `${ts} ${paint([ "bold", "magenta" ], "Slow query")} took 78ms: ${paint("magentaBright", "SELECT * FROM notes")}`
+ );
+ expect(colorizeLogEntry("Slow recursive query took 45ms.", "12:34:56.789")).toBe(
+ `${ts} ${paint([ "bold", "magenta" ], "Slow recursive query")} took 45ms.`
+ );
+ });
+
+ it("colours a whole error entry red, stack trace included", () => {
+ for (const message of [ "ERROR: boom\n at foo (bar.js:1:1)", "JS Error: boom\r\nStack: at x" ]) {
+ expect(colorizeLogEntry(message, "12:34:56.789")).toBe(`${ts} ${paint("red", message)}`);
+ }
+ });
+
+ it("leaves other entries uncoloured", () => {
+ for (const message of [ "JS Info: hello", "Becca (note cache) load took 120ms", "[Script abc] 200 GET /x" ]) {
+ expect(colorizeLogEntry(message, "12:34:56.789")).toBe(`${ts} ${message}`);
+ }
+ });
+});
describe("ServerLogService console echo", () => {
it("writes the entry to the log file but does not echo it under vitest", () => {
@@ -42,6 +83,14 @@ describe("ServerLogService console echo", () => {
log.info("server started");
expect(consoleLog).toHaveBeenCalledWith("server started");
+
+ // A colour terminal gets the timestamped, coloured entry instead.
+ (log as unknown as { colorizeConsole: boolean }).colorizeConsole = true;
+ log.info("500 GET /api/tree with 1 bytes took 3ms");
+
+ expect(consoleLog).toHaveBeenLastCalledWith(expect.stringMatching(
+ /^\x1b\[90m\d{2}:\d{2}:\d{2}\.\d{3}\x1b\[39m .*\x1b\[31m.*500/
+ ));
} finally {
if (previous !== undefined) {
process.env.VITEST = previous;
@@ -89,3 +138,7 @@ function logRequest(log: ServerLogService, url: string) {
log.request(req, res, 1, 15);
}
+
+function paint(format: Parameters<typeof styleText>[0], text: string) {
+ return styleText(format, text, { validateStream: false });
+}
diff --git a/apps/server/src/log_provider.ts b/apps/server/src/log_provider.ts
index 4a30c69f86..ac709f8722 100644
--- a/apps/server/src/log_provider.ts
+++ b/apps/server/src/log_provider.ts
@@ -3,6 +3,7 @@ import type { Request, Response } from "express";
import fs from "fs";
import { EOL } from "os";
import path from "path";
+import { styleText } from "util";
import config, { LOGGING_DEFAULT_RETENTION_DAYS } from "./services/config.js";
import dataDir from "./services/data_dir.js";
@@ -21,6 +22,7 @@ const requestBlacklist = [
export default class ServerLogService extends FileBasedLogService {
private logFile: fs.WriteStream | undefined;
+ private colorizeConsole = supportsColor(process.stdout);
constructor() {
super();
@@ -56,7 +58,7 @@ export default class ServerLogService extends FileBasedLogService {
this.logFile?.write(entry);
}
- protected override echoToConsole(str: string): void {
+ protected override echoToConsole(str: string, timestamp: string): void {
// Under vitest every console call is forwarded to the main process over the worker RPC, and
// this suite emits tens of thousands of them — the hidden-subtree bootstrap alone logs a
// line per created note, for each of the ~60 spec files that build the app. A call still in
@@ -67,7 +69,7 @@ export default class ServerLogService extends FileBasedLogService {
if (process.env.VITEST) {
return;
}
- super.echoToConsole(str);
+ super.echoToConsole(this.colorizeConsole ? colorizeLogEntry(str, timestamp) : str, timestamp);
}
protected override readLogFile(fileName: string): string | null {
@@ -145,3 +147,66 @@ export default class ServerLogService extends FileBasedLogService {
this.info(`${timeMs >= 10 ? "Slow " : ""}${res.statusCode} ${req.method} ${req.url} with ${responseLength} bytes took ${timeMs}ms`);
}
}
+
+type Style = Parameters<typeof styleText>[0];
+
+// The styles approximate the `--log-*` palette of `trilium_log_highlighter.ts` in
+// `packages/codemirror`, and the patterns match the same entries; keep the two in step.
+const TIMESTAMP_STYLE: Style = "gray";
+const SLOW_STYLE: Style = [ "bold", "yellow" ];
+const VERB_STYLE: Style = [ "bold", "blueBright" ];
+const URL_STYLE: Style = "cyan";
+const QUERY_VERB_STYLE: Style = [ "bold", "magenta" ];
+const SQL_STYLE: Style = "magentaBright";
+const ERROR_STYLE: Style = "red";
+const STATUS_STYLES: Record<string, Style> = {
+ "2": [ "bold", "green" ],
+ "3": [ "bold", "gray" ],
+ "4": [ "bold", "yellow" ],
+ "5": [ "bold", "red" ]
+};
+
+const HTTP_REQUEST = /^(Slow )?(\d{3}) (GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS) (\S+)/;
+const SLOW_QUERY = /^(Slow (?:recursive )?query)( took \d+ms)(?:(: )(.+))?/;
+const ERROR_ENTRY = /^(?:JS Error|ERROR):/;
+
+/** Prefixes `message` with its `timestamp` and colours both the way the Backend Log note does. */
+export function colorizeLogEntry(message: string, timestamp: string): string {
+ return `${paint(TIMESTAMP_STYLE, timestamp)} ${colorizeMessage(message)}`;
+}
+
+function colorizeMessage(message: string): string {
+ const http = HTTP_REQUEST.exec(message);
+ if (http) {
+ const [ matched, slowPrefix, status, verb, url ] = http;
+ return [
+ slowPrefix ? `${paint(SLOW_STYLE, "Slow")} ` : "",
+ STATUS_STYLES[status[0]] ? paint(STATUS_STYLES[status[0]], status) : status,
+ ` ${paint(VERB_STYLE, verb)} ${paint(URL_STYLE, url)}`,
+ message.slice(matched.length)
+ ].join("");
+ }
+
+ const query = SLOW_QUERY.exec(message);
+ if (query) {
+ const [ matched, queryVerb, took, separator = "", sql ] = query;
+ return [
+ paint(QUERY_VERB_STYLE, queryVerb),
+ took,
+ separator,
+ sql ? paint(SQL_STYLE, sql) : "",
+ message.slice(matched.length)
+ ].join("");
+ }
+
+ return ERROR_ENTRY.test(message) ? paint(ERROR_STYLE, message) : message;
+}
+
+function paint(style: Style, text: string): string {
+ return styleText(style, text, { validateStream: false });
+}
+
+/** Asks `styleText` itself, so a TTY check, `NO_COLOR` and `FORCE_COLOR` all apply as Node rules. */
+function supportsColor(stream: NodeJS.WriteStream): boolean {
+ return styleText("red", "x", { stream }) !== "x";
+}
diff --git a/docs/User Guide/User Guide/Troubleshooting/Error logs/Backend (server) logs.md b/docs/User Guide/User Guide/Troubleshooting/Error logs/Backend (server) logs.md
index 7d91f66f9c..d9f09911d9 100644
--- a/docs/User Guide/User Guide/Troubleshooting/Error logs/Backend (server) logs.md
+++ b/docs/User Guide/User Guide/Troubleshooting/Error logs/Backend (server) logs.md
@@ -12,6 +12,12 @@ Interaction since v0.104.0:
Backend logs are stored on the file system. To find them, open the <a class="reference-link" href="../../Installation%20%26%20Setup/Data%20directory.md">Data directory</a>, go to the `log` subdirectory and find the latest log file, e.g. `trilium-2022-12-14.log`. 
+## In the terminal
+
+The server also prints every log entry to its standard output, which is what `docker logs` or `journalctl` show. When that output is a terminal, each entry starts with its time and is colored the same way as the backend log in the app: request status codes by class, slow SQL queries, and errors in red. Output to a file, a pipe or a container log stays plain text without the time, since those tools add their own.
+
+To turn the colors off in a terminal, set the `NO_COLOR` environment variable; to force them on elsewhere, set `FORCE_COLOR=1`.
+
## Reporting backend bugs
You can attach the whole file to the bug report (preferable) or open it and copy-paste only the last lines / lines you believe are relevant.
diff --git a/packages/trilium-core/src/services/file_based_log.ts b/packages/trilium-core/src/services/file_based_log.ts
index 8a10c4a7f7..71f860b372 100644
--- a/packages/trilium-core/src/services/file_based_log.ts
+++ b/packages/trilium-core/src/services/file_based_log.ts
@@ -166,17 +166,18 @@ export default abstract class FileBasedLogService extends LogService {
let millisSinceMidnight = Date.now() - this.todaysMidnight.getTime();
millisSinceMidnight = this.checkDateAndRotate(millisSinceMidnight);
- const entry = `${this.formatTime(millisSinceMidnight)} ${str}${this.eol}`;
- this.writeEntry(entry);
- this.echoToConsole(str);
+ const timestamp = this.formatTime(millisSinceMidnight);
+ this.writeEntry(`${timestamp} ${str}${this.eol}`);
+ this.echoToConsole(str, timestamp);
}
/**
* Mirrors an entry to stdout, which is how `docker logs`/journalctl surface server activity.
* Overridable so a runtime that captures stdout through a costly channel can opt out while
- * still writing the entry to the log file.
+ * still writing the entry to the log file. The `timestamp` is the one written to the file;
+ * stdout omits it because log collectors stamp lines themselves.
*/
- protected echoToConsole(str: string): void {
+ protected echoToConsole(str: string, _timestamp: string): void {
console.log(str);
}