Skip to content

A platform for community discussion. Free, open, simple.

DEV: Introduce bundle-splitting for plugins (#41698)

David Taylor29487ba0bf22
diff --git a/app/views/layouts/_plugin_js.html.erb b/app/views/layouts/_plugin_js.html.erb
index 301d72cc90e..2d3c01bc415 100644
--- a/app/views/layouts/_plugin_js.html.erb
+++ b/app/views/layouts/_plugin_js.html.erb
@@ -18,10 +18,10 @@
<% end %>
<%- end %>
- <%- plugin_assets.flat_map { it[:imports] }.compact.uniq.each do |import| %>
+ <%- plugin_assets.flat_map { [*it[:imports], it[:route_bundle]] }.compact.uniq.each do |preload| %>
<link
rel="modulepreload"
- href="<%= script_asset_path(import) %>"
+ href="<%= script_asset_path(preload) %>"
nonce="<%= csp_nonce_placeholder %>"
>
<%- end %>
diff --git a/docs/developer-guides/docs/03-code-internals/31-bundle-splitting-for-plugins.md b/docs/developer-guides/docs/03-code-internals/31-bundle-splitting-for-plugins.md
new file mode 100644
index 00000000000..f32b858b62b
--- /dev/null
+++ b/docs/developer-guides/docs/03-code-internals/31-bundle-splitting-for-plugins.md
@@ -0,0 +1,146 @@
+---
+title: Bundle splitting for plugins
+short_title: Bundle splitting
+id: bundle-splitting
+---
+
+By default, every javascript module in a plugin is loaded when Discourse boots. This guide describes the `staticModules` system, which allows code to be automatically loaded on-demand, thereby improving initial boot times.
+
+This system is currently available to plugins. Theme support will come later
+
+## Enable static modules
+
+Add a `frontend` section to your plugin's `about.json`:
+
+```json
+{
+ "frontend": {
+ "staticModules": true
+ }
+}
+```
+
+Two things happen:
+
+Your components, helpers, modifiers and lib code stop being loaded automatically. Instead, they are only loaded when referenced by an import in other code.
+
+Your routes, controllers & templates move into a separate bundle which loads on demand.
+
+## Group your routes
+
+With `staticModules` on, all the routes in your plugin will go into one bundle. That is usually what you want.
+
+To split further, name a bundle on a route. The route and everything under it go in that bundle:
+
+```js
+export default function () {
+ this.route("chat", { bundleName: "chat" }, function () {
+ this.route("channel", { path: "/c/:title/:id" });
+ });
+
+ this.route("chat-settings", { bundleName: "chat-settings" });
+}
+```
+
+Here `chat` and `chat.channel` load together, and `chat-settings` loads separately.
+
+If a map mounts on another route and all of its routes belong in the same bundle, name it once on the map instead:
+
+```js
+export default {
+ resource: "admin.adminPlugins.show",
+ bundleName: "workflows-admin",
+
+ map() {
+ this.route("workflows", { path: "workflows" });
+ this.route("workflows-variables", { path: "variables" });
+ },
+};
+```
+
+## Manually loading code on-demand
+
+Bundling is automatic for routes only. Anything imported from an initializer or service is loaded eagerly.
+
+When something is large and rarely used (e.g. a modal that opens on a click), you should import it dynamically instead.
+
+```js
+export default async function showNewMessageModal(modal) {
+ const { default: ChatModalNewMessage } =
+ await import("../components/chat/modal/new-message");
+
+ modal.show(ChatModalNewMessage);
+}
+```
+
+The path must be a literal, so that the build can find it. To pull in a group of modules with one import, re-export them from a single module and import that.
+
+When lazy-loading components, it normally makes sense to use the `DAsyncContent` wrapper from `ui-kit`. For example:
+
+```gjs
+export default class LazyLoader extends Component {
+ get complexComponent() {
+ return import("./complex-component");
+ }
+
+ <template>
+ <DAsyncContent @asyncData={{this.complexComponent}}>
+ <:content as |componentModule|>
+ <componentModule.default />
+ </:content>
+ </DAsyncContent>
+ </template>
+}
+```
+
+`DAsyncContent` registers its own test waiter. If your plugin uses qunit tests and you call `import()` yourself, wrap the promise in `waitForPromise` so that tests wait for it to load:
+
+```js
+import { waitForPromise } from "@ember/test-waiters";
+
+await waitForPromise(import("./my-module"));
+```
+
+## Sharing code with other themes and plugins
+
+Enabling `staticModules` puts a boundary around your plugin. If another plugin or theme needs to import one of your modules, list it under `exports` in `about.json`:
+
+```json
+{
+ "frontend": {
+ "staticModules": true,
+ "exports": {
+ "discourse/components/chat-channel": "discourse/components/chat-channel",
+ "discourse/models/chat-channel": "discourse/models/chat-channel"
+ }
+ }
+}
+```
+
+The key is the name other plugins import by, and the value is the module behind it. Keeping the two apart means you can move a module without breaking anyone, and you can export the same module under more than one name. Other plugins reach these as `discourse/plugins/<your-plugin>/<key>`.
+
+Adding a module here will cause it to be eagerly loaded when the application boots, and therefore carries a performance cost. Consider providing an asynchronous-friendly entrypoint for any shared components/libs, so that most code is only loaded on the first use.
+
+## Route map syntax
+
+Route maps are statically analyzed during the build, so it's important for them to use simple syntax. Route maps must only include `this.route` calls, with literal string names and options.
+
+This means no conditionals, no loops, and no values read from settings, services, or imports. If the build cannot parse your map, your plugin will fail to compile.
+
+## Compatibility
+
+When using staticModules, you should avoid using these legacy features:
+
+- `templateName`/`controllerName` overrides in routes
+
+- Injecting controllers or routes via `@controller` or `.lookup()`
+
+- Loading components via `.lookup()`
+
+- `requirejs.entries`, `require()`, `define()`
+
+- Loading your plugin's modules from other plugins (except those listed in `exports`)
+
+## Verification
+
+Generated bundles are listed in `app/assets/generated/<your-plugin>/manifest.json` after a build. Discourse will preload the right bundle when a user lands directly on your page.
diff --git a/frontend/asset-processor/asset-processor-rollup.js b/frontend/asset-processor/asset-processor-rollup.js
index 699e2abb716..2cb2d087f2e 100644
--- a/frontend/asset-processor/asset-processor-rollup.js
+++ b/frontend/asset-processor/asset-processor-rollup.js
@@ -19,8 +19,12 @@ import discourseExternalLoader from "./rollup-plugins/discourse-external-loader"
import discourseFileSearch from "./rollup-plugins/discourse-file-search";
import discourseGjs from "./rollup-plugins/discourse-gjs";
import discourseHbs from "./rollup-plugins/discourse-hbs";
+import discourseRegisterComponents from "./rollup-plugins/discourse-register-components";
+import discourseRouteMaps from "./rollup-plugins/discourse-route-maps";
import discourseTerser from "./rollup-plugins/discourse-terser";
import discourseVirtualLoader from "./rollup-plugins/discourse-virtual-loader";
+import { labelFor, routeNamesFor } from "./rollup-virtual-imports";
+import { urlTableFor } from "./route-map-parser";
import buildEmberTemplateManipulatorPlugin from "./theme-hbs-ast-transforms";
import transformActionSyntax from "./transform-action-syntax";
import createVirtualFs from "./virtual-fs";
@@ -40,6 +44,10 @@ async function performRollup(modules, opts) {
const fs = createVirtualFs(modules, basePath);
+ // Filled in by `discourse-route-maps` before the entrypoint is generated from it.
+ const routeTables = { bundleByRoute: {}, derived: [] };
+ opts.routeTables = routeTables;
+
const cache = opts.pluginName ? caches.get(opts.pluginName) : false;
const result = await rollup({
@@ -55,6 +63,12 @@ async function performRollup(modules, opts) {
console.info(level, message);
},
plugins: [
+ discourseRouteMaps({
+ modules,
+ label: labelFor(opts),
+ tables: routeTables,
+ staticModules: !!opts.frontendConfig?.staticModules,
+ }),
discourseSourceImports({ fs }),
discourseFileSearch(),
discourseVirtualLoader({
@@ -119,6 +133,7 @@ async function performRollup(modules, opts) {
}),
discourseHbs(),
discourseGjs(),
+ discourseRegisterComponents({ basePath }),
discourseTerser({ opts }),
],
});
@@ -144,6 +159,55 @@ async function performRollup(modules, opts) {
),
];
+ const routeVirtualPrefix = `${basePath}virtual:route:`;
+
+ // Keyed by the qualified `<entrypoint>:<bundle>` from the module id, because two entrypoints
+ // can each have a bundle of the same name.
+ const bundleNameByFile = {};
+ for (const chunk of bundle.output) {
+ if (chunk.facadeModuleId?.startsWith(routeVirtualPrefix)) {
+ bundleNameByFile[chunk.fileName] = chunk.facadeModuleId.slice(
+ routeVirtualPrefix.length
+ );
+ }
+ }
+
+ // Rollup may hoist the entrypoint module into a shared chunk, so find where it landed.
+ function routeBundlesForEntry(entryName) {
+ const entrypointModuleId = `${basePath}virtual:entrypoint:${entryName}`;
+
+ const owner = bundle.output.find((chunk) =>
+ chunk.moduleIds?.includes(entrypointModuleId)
+ );
+
+ const fileNameByBundle = {};
+
+ for (const fileName of owner?.dynamicImports ?? []) {
+ const qualified = bundleNameByFile[fileName];
+
+ if (qualified?.startsWith(`${entryName}:`)) {
+ fileNameByBundle[qualified.slice(entryName.length + 1)] = fileName;
+ }
+ }
+
+ // A bundle name means a different bundle in each entrypoint.
+ const owned = routeNamesFor(
+ opts.entrypoints[entryName].modules,
+ routeTables.bundleByRoute
+ );
+ const urls = urlTableFor(
+ routeTables.derived.filter((route) => owned.has(route.name))
+ );
+
+ // Already ordered most specific first, which is the order Ruby matches them in.
+ return urls
+ .filter(({ bundleName }) => fileNameByBundle[bundleName])
+ .map(({ bundleName, url }) => ({
+ url,
+ fileName: fileNameByBundle[bundleName],
+ }));
+ }
+
const chunks = Object.fromEntries(
bundle.output
.filter((c) => c.code)
@@ -158,6 +222,9 @@ async function performRollup(modules, opts) {
imports: chunk.imports.filter((i) =>
bundle.output.find((c) => c.fileName === i)
),
+ routeBundles: chunk.isEntry
+ ? routeBundlesForEntry(chunk.name)
+ : undefined,
externalPluginImports,
},
];
diff --git a/frontend/asset-processor/rollup-plugins/discourse-register-components.js b/frontend/asset-processor/rollup-plugins/discourse-register-components.js
new file mode 100644
index 00000000000..d90a1070209
--- /dev/null
+++ b/frontend/asset-processor/rollup-plugins/discourse-register-components.js
@@ -0,0 +1,89 @@
+import MagicString from "magic-string";
+import { stripExtension } from "../rollup-virtual-imports";
+
+const STORE = "discourse/lib/deferred-class-modifications";
+const BINDING = "__discourseComponentClass";
+
+// What the resolver's suffix lookup accepts: `components/<name>` at any depth, and never a
+// module under `templates/`. `lookupModuleBySuffix` also falls back to `<name>/index`.
+const COMPONENT_REGEX = /(^|\/)components\/(.+)$/;
+const CLASSIC_TEMPLATE_REGEX = /(^|\/)templates\//;
+
+function componentPathFor(id, basePath) {
+ if (!id.startsWith(basePath)) {
+ return null;
+ }
+
+ const path = id.slice(basePath.length);
+
+ if (CLASSIC_TEMPLATE_REGEX.test(path)) {
+ return null;
+ }
+
+ const match = path.match(COMPONENT_REGEX);
+
+ return match ? stripExtension(match[2]).replace(/\/index$/, "") : null;
+}
+
+// Components are reached by import, so nothing can look one up by name. Each module says what
+// it is as it evaluates, which is when a `modifyClass` call that arrived first can be applied.
+export default function discourseRegisterComponents({ basePath }) {
+ return {
+ name: "discourse-register-components",
+
+ transform: {
+ // After the gjs and babel transforms, so this parses plain JS.
+ order: "post",
+
+ handler(code, id) {
+ const path = componentPathFor(id, basePath);
+
+ if (!path) {
+ return null;
+ }
+
+ const exported = this.parse(code).body.find(
+ (node) => node.type === "ExportDefaultDeclaration"
+ );
+
+ if (!exported) {
+ return null;
+ }
+
+ const declaration = exported.declaration;
+ const magic = new MagicString(code);
+
+ let binding;
+
+ if (declaration.type === "Identifier") {
+ binding = declaration.name;
+ } else if (declaration.type === "ClassDeclaration" && declaration.id) {
+ binding = declaration.id.name;
+ } else {
+ binding = BINDING;
+ const terminated = code[exported.end - 1] === ";";
+
+ magic.overwrite(
+ exported.start,
+ declaration.start,
+ `const ${binding} = `
+ );
+ magic.appendLeft(
+ exported.end,
+ `${terminated ? "" : ";"}\nexport default ${binding};`
+ );
+ }
+
+ magic.append(
+ `\nimport { registerModuleForModifyClass } from "${STORE}";\n` +
+ `registerModuleForModifyClass(${JSON.stringify(path)}, ${binding});\n`
+ );
+
+ return {
+ code: magic.toString(),
+ map: magic.generateMap({ hires: true }),
+ };
+ },
+ },
+ };
+}
diff --git a/frontend/asset-processor/rollup-plugins/discourse-register-components.test.mjs b/frontend/asset-processor/rollup-plugins/discourse-register-components.test.mjs
new file mode 100644
index 00000000000..68ecf6aa537
--- /dev/null
+++ b/frontend/asset-processor/rollup-plugins/discourse-register-components.test.mjs
@@ -0,0 +1,103 @@
+import { parseAst } from "rolldown/parseAst";
+import { describe, expect, it } from "vitest";
+import discourseRegisterComponents from "./discourse-register-components";
+
+const basePath = "discourse/plugins/chat/";
+
+// Stands in for the `this` a transform hook gets.
+const context = { parse: parseAst };
+
+function transform(code, file) {
+ const { handler } = discourseRegisterComponents({ basePath }).transform;
+ return handler.call(context, code, `${basePath}${file}`)?.code ?? null;
+}
+
+describe("discourse-register-components", () => {
+ it("registers a named default class under its resolver path", () => {
+ const out = transform(
+ `export default class ChatHeader {}`,
+ "discourse/components/chat-header.gjs"
+ );
+
+ expect(out).toContain(
+ `registerModuleForModifyClass("chat-header", ChatHeader);`
+ );
+ expect(out).toContain(`export default class ChatHeader {}`);
+ });
+
+ it("gives an anonymous default export a binding to register", () => {
+ const out = transform(
+ `import Component from "@glimmer/component";\nexport default class extends Component {}`,
+ "discourse/components/chat-header.gjs"
+ );
+
+ expect(out).toContain(`const __discourseComponentClass = class extends`);
+ expect(out).toContain(`export default __discourseComponentClass;`);
+ expect(out).toContain(
+ `registerModuleForModifyClass("chat-header", __discourseComponentClass);`
+ );
+ });
+
+ it("handles a default export that is just an identifier", () => {
+ const out = transform(
+ `const Old = 1;\nexport default Old;`,
+ "discourse/components/old-title.gjs"
+ );
+
+ expect(out).toContain(`registerModuleForModifyClass("old-title", Old);`);
+ });
+
+ it("uses the nested path, and drops an index segment", () => {
+ expect(
+ transform(
+ `export default class A {}`,
+ "discourse/components/chat/header/icon.gjs"
+ )
+ ).toContain(`"chat/header/icon"`);
+
+ expect(
+ transform(
+ `export default class A {}`,
+ "discourse/components/channel-icon/index.gjs"
+ )
+ ).toContain(`"channel-icon"`);
+ });
+
+ it("registers a component wherever the plugin nests it", () => {
+ // The resolver finds `components/<name>` as a path suffix at any depth.
+ expect(
+ transform(`export default class A {}`, "components/survey.gjs")
+ ).toContain(`"survey"`);
+ expect(
+ transform(
+ `export default class A {}`,
+ "discourse/app/components/panel.gjs"
+ )
+ ).toContain(`"panel"`);
+ });
+
+ it("leaves everything that is not a component alone", () => {
+ expect(
+ transform(`export default class A {}`, "discourse/routes/chat/channel.js")
+ ).toBe(null);
+ expect(
+ transform(`export const a = 1;`, "discourse/components/no-default.js")
+ ).toBe(null);
+ });
+
+ it("leaves classic component templates alone", () => {
+ // The resolver's suffix trie skips anything under `templates/`.
+ expect(
+ transform(
+ `export default class A {}`,
+ "discourse/templates/components/legacy.hbs"
+ )
+ ).toBe(null);
+ expect(
+ transform(
+ `export default class A {}`,
+ "discourse/templates/foo/components/bar.hbs"
+ )
+ ).toBe(null);
+ });
+});
diff --git a/frontend/asset-processor/rollup-plugins/discourse-route-maps.js b/frontend/asset-processor/rollup-plugins/discourse-route-maps.js
new file mode 100644
index 00000000000..fdaaf8d023e
--- /dev/null
+++ b/frontend/asset-processor/rollup-plugins/discourse-route-maps.js
@@ -0,0 +1,81 @@
+import {
+ buildRouteTree,
+ bundleByRouteFor,
+ deriveRoutes,
+ parseRouteMap,
+} from "../route-map-parser";
+
+// Added to the tree by `Plugin::JsManager`. These build routes in loops, so they are read
+// leniently and a plugin's `resource` can mount on them.
+const CORE_MAP_REGEX = /^__core__\//;
+
+const ROUTE_MAP_REGEX = /route-map\.js$/;
+
+function isCoreMap(filename) {
+ return CORE_MAP_REGEX.test(filename);
+}
+
+// Fills `tables` before any module loads, because the entrypoint's source is generated from it.
+export default function discourseRouteMaps({
+ modules,
+ label,
+ tables,
+ staticModules,
+}) {
+ return {
+ name: "discourse-route-maps",
+ buildStart() {
+ if (!staticModules) {
+ return;
+ }
+
+ const filenames = Object.keys(modules).filter((filename) =>
+ ROUTE_MAP_REGEX.test(filename)
+ );
+
+ // Themes get no core maps, so they cannot resolve a `resource` mount and derive no bundles.
+ if (!filenames.some(isCoreMap)) {
+ return;
+ }
+
+ // Core first, so a plugin extending a core route merges into it rather than creating it.
+ filenames.sort(
+ (a, b) =>
+ Number(!isCoreMap(a)) - Number(!isCoreMap(b)) || a.localeCompare(b)
+ );
+
+ const maps = filenames.map((filename) => {
+ const source = modules[filename];
+ const core = isCoreMap(filename);
+
+ return {
+ filename,
+ core,
+ ...parseRouteMap(this.parse(source), {
+ filename,
+ source,
+ label,
+ lenient: core,
+ }),
+ };
+ });
+
+ const { root, unmounted } = buildRouteTree(maps);
+
+ const [unmountable] = unmounted;
+
+ if (unmountable) {
+ throw new Error(
+ `[${label}] ${unmountable.filename} mounts on "${unmountable.resource}", which is not a route.`
+ );
+ }
+
+ const derived = deriveRoutes(root);
+
+ Object.assign(tables, {
+ derived,
+ bundleByRoute: bundleByRouteFor(derived),
+ });
+ },
+ };
+}
diff --git a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js
index 768e19b110d..d2bbd79d2b4 100644
--- a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js
+++ b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js
@@ -31,6 +31,7 @@ export default function discourseVirtualLoader({
? rollupVirtualImports
: {
"virtual:entrypoint": rollupVirtualImports["virtual:entrypoint"],
+ "virtual:route": rollupVirtualImports["virtual:route"],
};
return {
@@ -47,7 +48,8 @@ export default function discourseVirtualLoader({
if (
availableVirtualImports[source] ||
- source.startsWith("virtual:entrypoint:")
+ source.startsWith("virtual:entrypoint:") ||
+ source.startsWith("virtual:route:")
) {
return `${basePath}${source}`;
}
@@ -70,11 +72,32 @@ export default function discourseVirtualLoader({
return availableVirtualImports["virtual:entrypoint"](
entrypointConfig.modules,
opts,
- {
- basePath,
- context: this,
- }
+ entrypointName
);
+ } else if (fromBase.startsWith("virtual:route:")) {
+ // Entrypoints are built from separate module lists but share one bundle-name
+ // namespace, so the entrypoint is part of the id.
+ const qualified = fromBase.replace("virtual:route:", "");
+ const separator = qualified.indexOf(":");
+ const entrypointName = qualified.slice(0, separator);
+ const bundleName = qualified.slice(separator + 1);
+ const entrypointConfig = entrypoints[entrypointName];
+
+ const source =
+ entrypointConfig &&
+ availableVirtualImports["virtual:route"](
+ entrypointConfig.modules,
+ opts,
+ bundleName
+ );
+
+ if (!source) {
+ throw new Error(
+ `No route bundle for "${bundleName}" in the ${entrypointName} entrypoint — no route files matched it.`
+ );
+ }
+
+ return source;
} else if (availableVirtualImports[fromBase]) {
return availableVirtualImports[fromBase](opts);
}
diff --git a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs
index 05127c4642c..3008918cffa 100644
--- a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs
+++ b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs
@@ -6,11 +6,11 @@ const basePath = "discourse/plugins/chat/";
const PLUGIN_API_WRAPPER = `\0${basePath}virtual:attributed-plugin-api`;
const API_INITIALIZER_WRAPPER = `\0${basePath}virtual:attributed-api-initializer`;
-function loader(isTheme = false) {
+function loader(isTheme = false, entrypoints = {}, opts = {}) {
return discourseVirtualLoader({
basePath,
- entrypoints: {},
- opts: { pluginName: "chat" },
+ entrypoints,
+ opts: { pluginName: "chat", ...opts },
isTheme,
});
}
@@ -54,3 +54,44 @@ test("plugin builds can load the wrappers but not virtual:theme", () => {
expect(load(PLUGIN_API_WRAPPER)).toContain("_INTERNAL_SOURCE_KEY");
expect(load(`${basePath}virtual:theme`)).toBeUndefined();
});
+
+test("loads a route bundle from its own entrypoint, not another with the same name", () => {
+ const { load } = loader(
+ false,
+ {
+ main: { modules: ["discourse/routes/chat.js"] },
+ admin: { modules: ["discourse/routes/admin-plugins/show/hooks.js"] },
+ },
+ {
+ frontendConfig: { staticModules: true },
+ routeTables: {
+ bundleByRoute: {
+ chat: "default",
+ "admin-plugins.show.hooks": "default",
+ },
+ },
+ }
+ );
+
+ expect(load(`${basePath}virtual:route:main:default`)).toContain(
+ '"discourse/routes/chat":'
+ );
+ expect(load(`${basePath}virtual:route:admin:default`)).toContain(
+ '"discourse/routes/admin-plugins/show/hooks":'
+ );
+});
+
+test("a route bundle missing from the named entrypoint is an error", () => {
+ const { load } = loader(
+ false,
+ { main: { modules: ["discourse/routes/chat.js"] } },
+ {
+ frontendConfig: { staticModules: true },
+ routeTables: { bundleByRoute: { chat: "default" } },
+ }
+ );
+
+ expect(() => load(`${basePath}virtual:route:admin:default`)).toThrow(
+ /No route bundle for "default" in the admin entrypoint/
+ );
+});
diff --git a/frontend/asset-processor/rollup-virtual-imports.js b/frontend/asset-processor/rollup-virtual-imports.js
index f185575235a..81360c55cd3 100644
--- a/frontend/asset-processor/rollup-virtual-imports.js
+++ b/frontend/asset-processor/rollup-virtual-imports.js
@@ -9,79 +9,299 @@ const SUPPORTED_FILE_EXTENSIONS = [
const IS_CONNECTOR_REGEX = /(^|\/)connectors\//;
-export default {
- "virtual:entrypoint": (moduleFilenames, { themeId, pluginName }) => {
- const label = pluginName ? `PLUGIN ${pluginName}` : `THEME ${themeId}`;
- const imports = [];
- const entries = [];
- const warnings = [];
-
- const exportedModules = new Set();
-
- let i = 1;
- for (const moduleFilename of moduleFilenames) {
- // Type-only declaration files have no runtime module to export.
- if (moduleFilename.endsWith(".d.ts")) {
- continue;
- }
+// Scanned by name at runtime, at any depth, so these have to stay registered with `define()`.
+const EAGER_DIRECTORIES = [
+ "connectors",
+ "services",
+ "models",
+ "adapters",
+ "discourse-markdown",
+ "markdown-it",
+ "pre-initializers",
+ "initializers",
+ "api-initializers",
+ "instance-initializers",
+];
+
+const EAGER_DIRECTORY_REGEX = new RegExp(
+ `(^|/)(${EAGER_DIRECTORIES.join("|")})/`
+);
+
+function isEagerModule(compatModuleName) {
+ return (
+ EAGER_DIRECTORY_REGEX.test(compatModuleName) ||
+ // `mapRoutes` matches on the suffix alone.
+ /route-map$/.test(compatModuleName)
+ );
+}
+
+export function stripExtension(filename) {
+ return filename.replace(/\.[^\.]+(\.es6)?$/, "");
+}
+
+function normalizeModules(moduleFilenames, label) {
+ const records = [];
+ const warnings = [];
+ const seen = new Set();
+
+ for (const moduleFilename of moduleFilenames) {
+ if (moduleFilename.endsWith(".d.ts")) {
+ continue;
+ }
- if (
- !SUPPORTED_FILE_EXTENSIONS.some((ext) => moduleFilename.endsWith(ext))
- ) {
- // Unsupported file type. Log a warning and skip
- warnings.push(
- `console.warn("[${label}] Unsupported file type: ${moduleFilename}");`
+ if (
+ !SUPPORTED_FILE_EXTENSIONS.some((ext) => moduleFilename.endsWith(ext))
+ ) {
+ warnings.push(
+ `console.warn("[${label}] Unsupported file type: ${moduleFilename}");`
+ );
+ continue;
+ }
+
+ const filenameWithoutExtension = stripExtension(moduleFilename);
+
+ let compatModuleName = filenameWithoutExtension;
+
+ if (moduleFilename.match(IS_CONNECTOR_REGEX)) {
+ const isTemplate = moduleFilename.endsWith(".hbs");
+ const isInTemplatesDirectory = moduleFilename.match(/(^|\/)templates\//);
+
+ if (isTemplate && !isInTemplatesDirectory) {
+ compatModuleName = compatModuleName.replace(
+ IS_CONNECTOR_REGEX,
+ "$1templates/connectors/"
);
- continue;
+ } else if (!isTemplate && isInTemplatesDirectory) {
+ compatModuleName = compatModuleName.replace(/(^|\/)templates\//, "$1");
}
+ }
- const filenameWithoutExtension = moduleFilename.replace(
- /\.[^\.]+(\.es6)?$/,
- ""
- );
+ const importPath = filenameWithoutExtension.match(IS_CONNECTOR_REGEX)
+ ? moduleFilename
+ : filenameWithoutExtension;
+
+ if (seen.has(importPath)) {
+ continue;
+ }
+ seen.add(importPath);
+
+ records.push({ importPath, compatModuleName });
+ }
+
+ return { records, warnings };
+}
+
+// Anchored to the top-level segment: a component under `components/chat/routes/` is not a route.
+const ROUTE_FILE_REGEX = /^[^/]+\/(routes|controllers|templates)\/(.+)$/;
+
+function routeNameFor(compatModuleName) {
+ const match = compatModuleName.match(ROUTE_FILE_REGEX);
+
+ if (!match) {
+ return null;
+ }
+
+ const [, type, path] = match;
+
+ // Discourse nests connectors and classic component templates under `templates/` too.
+ if (
+ type === "templates" &&
+ (path.startsWith("connectors/") || path.startsWith("components/"))
+ ) {
+ return null;
+ }
+
+ return path.split("/").join(".");
+}
+
+// Ember creates these without a `this.route` call, so no route map names them.
+const IMPLICIT_ROUTE_SUFFIXES = ["index", "loading", "error"];
+
+function bundleNameFor(routeName, bundleByRoute) {
+ let name = routeName;
+
+ while (name) {
+ if (bundleByRoute[name]) {
+ return bundleByRoute[name];
+ }
+
+ const dot = name.lastIndexOf(".");
+
+ if (!IMPLICIT_ROUTE_SUFFIXES.includes(name.slice(dot + 1))) {
+ return null;
+ }
+
+ name = dot === -1 ? "" : name.slice(0, dot);
+ }
+
+ return null;
+}
- let compatModuleName = filenameWithoutExtension;
-
- if (moduleFilename.match(IS_CONNECTOR_REGEX)) {
- const isTemplate = moduleFilename.endsWith(".hbs");
- const isInTemplatesDirectory =
- moduleFilename.match(/(^|\/)templates\//);
-
- if (isTemplate && !isInTemplatesDirectory) {
- compatModuleName = compatModuleName.replace(
- IS_CONNECTOR_REGEX,
- "$1templates/connectors/"
- );
- } else if (!isTemplate && isInTemplatesDirectory) {
- compatModuleName = compatModuleName.replace(
- /(^|\/)templates\//,
- "$1"
- );
+export function routeBundlesFor(records, bundleByRoute) {
+ if (!bundleByRoute) {
+ return [];
+ }
+
+ const bundles = new Map();
+
+ for (const record of records) {
+ const routeName = routeNameFor(record.compatModuleName);
+ const bundleName = routeName && bundleNameFor(routeName, bundleByRoute);
+
+ if (!bundleName) {
+ continue;
+ }
+
+ let bundle = bundles.get(bundleName);
+
+ if (!bundle) {
+ bundle = { bundleName, names: new Set(), records: [] };
+ bundles.set(bundleName, bundle);
+ }
+
+ bundle.names.add(routeName);
+ bundle.records.push(record);
+ }
+
+ return [...bundles.values()].map((bundle) => ({
+ ...bundle,
+ names: [...bundle.names].sort(),
+ }));
+}
+
+export function routeNamesFor(moduleFilenames, bundleByRoute) {
+ const { records } = normalizeModules(moduleFilenames);
+
+ return new Set(
+ routeBundlesFor(records, bundleByRoute).flatMap((bundle) => bundle.names)
+ );
+}
+
+export function labelFor({ pluginName, themeId }) {
+ return pluginName ? `PLUGIN ${pluginName}` : `THEME ${themeId}`;
+}
+
+function renderImports(records) {
+ const identifiers = new Map(
+ records.map((record, i) => [record, `Mod${i + 1}`])
+ );
+
+ const lines = records.map(
+ (record) =>
+ `import * as ${identifiers.get(record)} from "./${record.importPath}";`
+ );
+
+ return { lines, identifiers };
+}
+
+function renderMap(name, records, identifiers) {
+ return [
+ `const ${name} = {`,
+ ...records.map(
+ (record) => ` "${record.compatModuleName}": ${identifiers.get(record)},`
+ ),
+ "};",
+ ];
+}
+
+export default {
+ "virtual:entrypoint": (moduleFilenames, opts, entrypointName) => {
+ const { frontendConfig } = opts;
+
+ const { records, warnings } = normalizeModules(
+ moduleFilenames,
+ labelFor(opts)
+ );
+
+ // QUnit only finds a test by running the module that registers it, so tests stay eager.
+ if (entrypointName === "test" || !frontendConfig?.staticModules) {
+ const { lines, identifiers } = renderImports(records);
+
+ return [
+ ...lines,
+ ...warnings,
+ ...renderMap("compatModules", records, identifiers),
+ "export { compatModules };",
+ "export default compatModules;",
+ "",
+ ].join("\n");
+ }
+
+ // `exports` maps the name other bundles import by to the module behind it. That module can be
+ // named with or without the `/index` a colocated component adds.
+ const publicNamesByPath = new Map();
+
+ for (const [publicName, internalName] of Object.entries(
+ frontendConfig.exports ?? {}
+ )) {
+ const path = stripExtension(internalName);
+
+ for (const candidate of [path, `${path}/index`]) {
+ if (!publicNamesByPath.has(candidate)) {
+ publicNamesByPath.set(candidate, []);
}
+ publicNamesByPath.get(candidate).push(publicName);
}
+ }
- const importPath = filenameWithoutExtension.match(IS_CONNECTOR_REGEX)
- ? moduleFilename
- : filenameWithoutExtension;
+ const bundles = routeBundlesFor(records, opts.routeTables?.bundleByRoute);
- if (exportedModules.has(importPath)) {
- continue;
- }
- exportedModules.add(importPath);
+ const eager = records.filter((record) =>
+ isEagerModule(record.compatModuleName)
+ );
- imports.push(`import * as Mod${i} from "./${importPath}";`);
- entries.push(` "${compatModuleName}": Mod${i},`);
+ // One module can be exported under several names.
+ const exported = records.flatMap((record) =>
+ (publicNamesByPath.get(stripExtension(record.importPath)) ?? []).map(
+ (publicName) => ({ publicName, record })
+ )
+ );
- i += 1;
- }
+ const imported = [
+ ...new Set([...eager, ...exported.map(({ record }) => record)]),
+ ];
+ const { lines, identifiers } = renderImports(imported);
return [
- ...imports,
+ ...lines,
...warnings,
- "const compatModules = {",
- ...entries,
+ ...renderMap("compatModules", eager, identifiers),
+ "const pluginExports = {",
+ ...exported.map(
+ ({ publicName, record }) =>
+ ` "${publicName}": ${identifiers.get(record)},`
+ ),
"};",
- "export default compatModules;",
+ "export const routes = [",
+ ...bundles.map(
+ (bundle) =>
+ ` { names: ${JSON.stringify(bundle.names)},` +
+ ` load: () => import("virtual:route:${entrypointName}:${bundle.bundleName}") },`
+ ),
+ "];",
+ "export { compatModules };",
+ "export default pluginExports;",
+ "",
+ ].join("\n");
+ },
+ // The default export goes to `Resolver#addModules`, so it must be a plain module map.
+ "virtual:route": (moduleFilenames, opts, bundleName) => {
+ const { records } = normalizeModules(moduleFilenames, labelFor(opts));
+ const bundle = routeBundlesFor(
+ records,
+ opts.routeTables?.bundleByRoute
+ ).find((candidate) => candidate.bundleName === bundleName);
+
+ if (!bundle) {
+ return null;
+ }
+
+ const { lines, identifiers } = renderImports(bundle.records);
+
+ return [
+ ...lines,
+ ...renderMap("routeCompatModules", bundle.records, identifiers),
+ "export default routeCompatModules;",
"",
].join("\n");
},
diff --git a/frontend/asset-processor/rollup-virtual-imports.test.mjs b/frontend/asset-processor/rollup-virtual-imports.test.mjs
index df6f90175c0..d729813207f 100644
--- a/frontend/asset-processor/rollup-virtual-imports.test.mjs
+++ b/frontend/asset-processor/rollup-virtual-imports.test.mjs
@@ -1,44 +1,411 @@
-/* eslint-disable qunit/require-expect */
-import { expect, test } from "vitest";
-import { privateVirtualImports } from "./rollup-virtual-imports.js";
+import { describe, expect, it } from "vitest";
+import rollupVirtualImports, {
+ privateVirtualImports,
+ routeNamesFor,
+} from "./rollup-virtual-imports";
-test("attributed plugin-api module wraps withPluginApi for a plugin", () => {
- const code = privateVirtualImports["virtual:attributed-plugin-api"]({
- pluginName: "chat",
+function entrypoint(moduleFilenames, opts = {}, entrypointName = "main") {
+ return rollupVirtualImports["virtual:entrypoint"](
+ moduleFilenames,
+ {
+ pluginName: "chat",
+ ...opts,
+ },
+ entrypointName
+ );
+}
+
+const MODULES = [
+ "discourse/components/chat-channel.gjs",
+ "discourse/services/chat.js",
+ "discourse/models/channel.js",
+ "discourse/adapters/chat.js",
+ "discourse/initializers/chat-setup.js",
+ "discourse/api-initializers/chat.js",
+ "discourse/pre-initializers/chat-early.js",
+ "discourse/chat-route-map.js",
+ "discourse/connectors/user-menu/chat.gjs",
+ "discourse/routes/chat/channel.js",
+ "discourse/controllers/chat/channel.js",
+ "discourse/templates/chat/channel.hbs",
+ "discourse/lib/chat-utils.js",
+ "discourse/helpers/format-chat-date.js",
+];
+
+describe("virtual:entrypoint", () => {
+ describe("without staticModules", () => {
+ it("eagerly imports every module and exports one object under both names", () => {
+ const output = entrypoint(MODULES);
+
+ for (const filename of MODULES) {
+ // Connectors keep their extension; everything else is stripped.
+ const importPath = filename.includes("/connectors/")
+ ? filename
+ : filename.replace(/\.\w+$/, "");
+ expect(output, filename).toContain(`from "./${importPath}"`);
+ }
+
+ expect(output).toContain("export { compatModules };");
+ expect(output).toContain("export default compatModules;");
+ expect(output).not.toContain("pluginExports");
+ });
+
+ it("skips type declarations and warns about unsupported files", () => {
+ const output = entrypoint([
+ "discourse/lib/types.d.ts",
+ "discourse/lib/notes.md",
+ "discourse/services/chat.js",
+ ]);
+
+ expect(output).not.toContain("types.d.ts");
+ expect(output).toContain("Unsupported file type: discourse/lib/notes.md");
+ expect(output).toContain('from "./discourse/services/chat"');
+ });
});
- expect(code).toContain(
- 'import { _INTERNAL_SOURCE_KEY } from "discourse/lib/api";'
- );
- expect(code).toContain(
- 'import { withPluginApi as _withPluginApi } from "discourse/lib/plugin-api";'
- );
- expect(code).toContain('Object.freeze({"type":"plugin","name":"chat"})');
- expect(code).toContain("export function withPluginApi(...args)");
- expect(code).toContain("[_INTERNAL_SOURCE_KEY]: SOURCE");
- expect(code).toContain("return _withPluginApi(...args);");
-});
+ describe("with staticModules", () => {
+ const output = entrypoint(MODULES, {
+ frontendConfig: {
+ staticModules: true,
+ exports: {
+ "discourse/components/chat-channel":
+ "discourse/components/chat-channel.gjs",
+ "discourse/models/channel": "discourse/models/channel.js",
+ },
+ },
+ });
+
+ const compatModules = output.slice(
+ output.indexOf("const compatModules"),
+ output.indexOf("const pluginExports")
+ );
+ const pluginExports = output.slice(output.indexOf("const pluginExports"));
+
+ it("registers everything Discourse resolves by name", () => {
+ for (const name of [
+ "discourse/services/chat",
+ "discourse/models/channel",
+ "discourse/adapters/chat",
+ "discourse/initializers/chat-setup",
+ "discourse/api-initializers/chat",
+ "discourse/pre-initializers/chat-early",
+ "discourse/chat-route-map",
+ "discourse/connectors/user-menu/chat",
+ ]) {
+ expect(compatModules, name).toContain(`"${name}":`);
+ }
+ });
+
+ it("registers them wherever the plugin nests them", () => {
+ // The resolver matches a path suffix, and the connector and markdown scans match anywhere,
+ // so a plugin is free to keep these below the top level.
+ const nested = entrypoint(
+ [
+ "discourse/app/services/voice-webrtc.js",
+ "discourse/lib/discourse-markdown/spoiler.js",
+ "discourse-markdown/graphviz.js",
+ "pre-initializers/h-captcha.js",
+ "discourse/templates/connectors/user-menu/chat.hbs",
+ ],
+ { frontendConfig: { staticModules: true } }
+ );
+
+ for (const name of [
+ "discourse/app/services/voice-webrtc",
+ "discourse/lib/discourse-markdown/spoiler",
+ "discourse-markdown/graphviz",
+ "pre-initializers/h-captcha",
+ "discourse/templates/connectors/user-menu/chat",
+ ]) {
+ expect(nested, name).toContain(`"${name}":`);
+ }
+ });
+
+ it("does not register routes, controllers or route templates", () => {
+ for (const name of [
+ "discourse/routes/chat/channel",
+ "discourse/controllers/chat/channel",
+ "discourse/templates/chat/channel",
+ ]) {
+ expect(compatModules, name).not.toContain(`"${name}":`);
+ }
+ });
-test("attributed api-initializer module wraps apiInitializer for a theme", () => {
- const code = privateVirtualImports["virtual:attributed-api-initializer"]({
- themeId: 42,
+ it("leaves invokables and lib code to be statically imported", () => {
+ // Registering these would stop them tree-shaking.
+ expect(compatModules).not.toContain(
+ '"discourse/helpers/format-chat-date"'
+ );
+ expect(compatModules).not.toContain('"discourse/lib/chat-utils"');
+ expect(compatModules).not.toContain(
+ '"discourse/components/chat-channel"'
+ );
+ });
+
+ it("exports only the declared modules as the cross-bundle API", () => {
+ expect(pluginExports).toContain('"discourse/components/chat-channel":');
+ expect(pluginExports).toContain('"discourse/models/channel":');
+ expect(pluginExports).not.toContain('"discourse/services/chat":');
+ expect(output).toContain("export default pluginExports;");
+ });
+
+ it("keys the export by its public name, not the module behind it", () => {
+ const aliased = entrypoint(MODULES, {
+ frontendConfig: {
+ staticModules: true,
+ exports: {
+ "discourse/chat-channel": "discourse/components/chat-channel.gjs",
+ "discourse/channel": "discourse/components/chat-channel.gjs",
+ },
+ },
+ });
+ const map = aliased.slice(aliased.indexOf("const pluginExports"));
+
+ expect(map).toContain('"discourse/chat-channel":');
+ expect(map).toContain('"discourse/channel":');
+ expect(map).not.toContain('"discourse/components/chat-channel":');
+ });
+
+ it("imports a module shared and registered only once", () => {
+ // `discourse/models/channel` is both eager and shared.
+ const imports = output
+ .split("\n")
+ .filter((line) => line.includes('from "./discourse/models/channel"'));
+ expect(imports).toHaveLength(1);
+ });
+
+ it("does not import modules which are neither eager nor shared", () => {
+ expect(output).not.toContain('from "./discourse/lib/chat-utils"');
+ expect(output).not.toContain(
+ 'from "./discourse/helpers/format-chat-date"'
+ );
+ });
+
+ it("emits no routes export when nothing is split", () => {
+ expect(output).toContain("export const routes = [\n];");
+ });
});
- expect(code).toContain(
- 'import { apiInitializer as _apiInitializer } from "discourse/lib/api";'
- );
- expect(code).toContain('Object.freeze({"type":"theme","id":42})');
- expect(code).toContain("export function apiInitializer(...args)");
- expect(code).toContain("return _apiInitializer(...args);");
+ describe("test entrypoint", () => {
+ const TEST_MODULES = [
+ "acceptance/chat-message-test.gjs",
+ "components/chat-channel-test.gjs",
+ "unit/lib/chat-utils-test.js",
+ "unit/services/chat-test.js",
+ "integration/components/user-menu/chat-test.gjs",
+ "helpers/chat-fixtures.js",
+ ];
+
+ it("eagerly imports every module despite staticModules", () => {
+ const output = entrypoint(
+ TEST_MODULES,
+ { frontendConfig: { staticModules: true, exports: {} } },
+ "test"
+ );
+
+ for (const filename of TEST_MODULES) {
+ const importPath = filename.replace(/\.\w+$/, "");
+ expect(output, filename).toContain(`from "./${importPath}"`);
+ }
+
+ expect(output).toContain("export default compatModules;");
+ expect(output).not.toContain("pluginExports");
+ });
+
+ it("still tree-shakes the same modules for a non-test entrypoint", () => {
+ const output = entrypoint(
+ TEST_MODULES,
+ { frontendConfig: { staticModules: true, exports: {} } },
+ "main"
+ );
+
+ expect(output).not.toContain('from "./components/chat-channel-test"');
+ expect(output).not.toContain('from "./unit/lib/chat-utils-test"');
+ });
+ });
+
+ describe("route bundles", () => {
+ const ROUTE_MODULES = [
+ "discourse/routes/chat.js",
+ "discourse/routes/chat/channel.js",
+ "discourse/controllers/chat/channel.js",
+ "discourse/templates/chat/channel.hbs",
+ "discourse/routes/chat/visualizer.js",
+ "discourse/templates/chat/visualizer.hbs",
+ "discourse/routes/browse.js",
+ "discourse/services/chat.js",
+ "discourse/templates/connectors/user-menu/chat.hbs",
+ "discourse/templates/components/chat-message.hbs",
+ // Chat really has these: components in a directory called `routes`.
+ "discourse/components/chat/routes/channel.gjs",
+ ];
+
+ const frontendConfig = { staticModules: true };
+
+ // `chat.visualizer` names its own bundle, so it is not swept into `chat`.
+ const routeTables = {
+ bundleByRoute: {
+ chat: "chat",
+ "chat.channel": "chat",
+ "chat.visualizer": "visualizer",
+ },
+ };
+
+ const output = entrypoint(ROUTE_MODULES, { frontendConfig, routeTables });
+
+ it("groups routes into a bundle per declared name", () => {
+ expect(output).toContain(
+ `{ names: ["chat.visualizer"], load: () => import("virtual:route:main:visualizer") },`
+ );
+ expect(output).toContain(
+ `{ names: ["chat","chat.channel"], load: () => import("virtual:route:main:chat") },`
+ );
+ });
+
+ it("puts an implicit index route in its parent's bundle", () => {
+ const withIndex = entrypoint(
+ [...ROUTE_MODULES, "discourse/routes/chat/index.js"],
+ { frontendConfig, routeTables }
+ );
+
+ expect(withIndex).toContain(
+ `{ names: ["chat","chat.channel","chat.index"], load: () => import("virtual:route:main:chat") },`
+ );
+ });
+
+ it("keeps a bundle name in one entrypoint clear of the same name in another", () => {
+ // Both entrypoints put their unnamed routes in `default`, and each is built from its own
+ // module list.
+ const shared = { bundleByRoute: { chat: "default", admin: "default" } };
+
+ const main = entrypoint(
+ ["discourse/routes/chat.js"],
+ { frontendConfig, routeTables: shared },
+ "main"
+ );
+ const admin = entrypoint(
+ ["discourse/routes/admin.js"],
+ { frontendConfig, routeTables: shared },
+ "admin"
+ );
+
+ expect(main).toContain(
+ `{ names: ["chat"], load: () => import("virtual:route:main:default") },`
+ );
+ expect(admin).toContain(
+ `{ names: ["admin"], load: () => import("virtual:route:admin:default") },`
+ );
+ });
+
+ it("keeps split route files out of the eager set", () => {
+ const compatModules = output.slice(
+ output.indexOf("const compatModules"),
+ output.indexOf("const pluginExports")
+ );
+
+ expect(compatModules).not.toContain('"discourse/routes/chat"');
+ expect(compatModules).not.toContain('"discourse/routes/chat/channel"');
+ expect(compatModules).not.toContain('"discourse/templates/chat/channel"');
+
+ // A route no map names is not registered either.
+ expect(compatModules).not.toContain('"discourse/routes/browse":');
+
+ expect(compatModules).toContain('"discourse/services/chat":');
+ });
+
+ it("does not mistake connectors or component templates for routes", () => {
+ expect(output).not.toContain('import("virtual:route:connectors');
+ expect(output).not.toContain('import("virtual:route:components');
+ expect(output).not.toContain('"connectors.user-menu.chat"');
+ expect(output).not.toContain('"components.chat-message"');
+ });
+
+ it("only treats top-level routes/controllers/templates as routes", () => {
+ const compatModules = output.slice(
+ output.indexOf("const compatModules"),
+ output.indexOf("const pluginExports")
+ );
+
+ expect(compatModules).not.toContain(
+ '"discourse/components/chat/routes/channel"'
+ );
+ expect(output).not.toContain('import("virtual:route:channel")');
+ });
+
+ it("renders a route bundle as a plain module map", () => {
+ const bundle = rollupVirtualImports["virtual:route"](
+ ROUTE_MODULES,
+ { pluginName: "chat", frontendConfig, routeTables },
+ "chat"
+ );
+
+ expect(bundle).toContain('"discourse/routes/chat":');
+ expect(bundle).toContain('"discourse/routes/chat/channel":');
+ expect(bundle).toContain('"discourse/controllers/chat/channel":');
+ expect(bundle).toContain('"discourse/templates/chat/channel":');
+ expect(bundle).toContain("export default routeCompatModules;");
+
+ // The separately-split child does not belong to the parent bundle.
+ expect(bundle).not.toContain('"discourse/routes/chat/visualizer":');
+ });
+ });
});
-test("the source lands on opts, after the legacy version argument", () => {
- const code = privateVirtualImports["virtual:attributed-plugin-api"]({
- pluginName: "chat",
+describe("privateVirtualImports", () => {
+ it("wraps withPluginApi for a plugin", () => {
+ const code = privateVirtualImports["virtual:attributed-plugin-api"]({
+ pluginName: "chat",
+ });
+
+ expect(code).toContain(
+ 'import { _INTERNAL_SOURCE_KEY } from "discourse/lib/api";'
+ );
+ expect(code).toContain(
+ 'import { withPluginApi as _withPluginApi } from "discourse/lib/plugin-api";'
+ );
+ expect(code).toContain('Object.freeze({"type":"plugin","name":"chat"})');
+ expect(code).toContain("export function withPluginApi(...args)");
+ expect(code).toContain("[_INTERNAL_SOURCE_KEY]: SOURCE");
+ expect(code).toContain("return _withPluginApi(...args);");
});
- expect(code).toContain('typeof args[0] === "string" ? 2 : 1');
- expect(code).toContain(
- "args[optsIndex] = { ...args[optsIndex], [_INTERNAL_SOURCE_KEY]: SOURCE };"
- );
+ it("wraps apiInitializer for a theme", () => {
+ const code = privateVirtualImports["virtual:attributed-api-initializer"]({
+ themeId: 42,
+ });
+
+ expect(code).toContain(
+ 'import { apiInitializer as _apiInitializer } from "discourse/lib/api";'
+ );
+ expect(code).toContain('Object.freeze({"type":"theme","id":42})');
+ expect(code).toContain("export function apiInitializer(...args)");
+ expect(code).toContain("return _apiInitializer(...args);");
+ });
+
+ it("puts the source on opts, after the legacy version argument", () => {
+ const code = privateVirtualImports["virtual:attributed-plugin-api"]({
+ pluginName: "chat",
+ });
+
+ expect(code).toContain('typeof args[0] === "string" ? 2 : 1');
+ expect(code).toContain(
+ "args[optsIndex] = { ...args[optsIndex], [_INTERNAL_SOURCE_KEY]: SOURCE };"
+ );
+ });
+});
+
+describe("routeNamesFor", () => {
+ it("reports only the routes a module list backs", () => {
+ const bundleByRoute = { chat: "default", "admin.hooks": "default" };
+
+ expect([
+ ...routeNamesFor(["discourse/routes/chat.js"], bundleByRoute),
+ ]).toEqual(["chat"]);
+ expect([
+ ...routeNamesFor(["discourse/templates/admin/hooks.hbs"], bundleByRoute),
+ ]).toEqual(["admin.hooks"]);
+ expect([
+ ...routeNamesFor(["discourse/services/chat.js"], bundleByRoute),
+ ]).toEqual([]);
+ });
});
diff --git a/frontend/asset-processor/route-map-parser.js b/frontend/asset-processor/route-map-parser.js
new file mode 100644
index 00000000000..8797c3bd325
--- /dev/null
+++ b/frontend/asset-processor/route-map-parser.js
@@ -0,0 +1,358 @@
+// Route maps are parsed, never run: plugin and theme code is not trusted enough to evaluate here.
+// `mapping-router.js` is the reference for what a map means.
+
+// `BareRouter#lazyRoute` dasherizes before matching, so derived names have to agree.
+function dasherize(value) {
+ return value
+ .replace(/([a-z\d])([A-Z])/g, "$1-$2")
+ .replace(/_/g, "-")
+ .toLowerCase();
+}
+
+function positionOf(source, offset) {
+ const upTo = source.slice(0, offset);
+ const line = upTo.split("\n").length;
+ return { line, column: offset - (upTo.lastIndexOf("\n") + 1) + 1 };
+}
+
+function fail(context, node, message) {
+ const { line, column } = positionOf(context.source, node.start);
+ throw new Error(
+ `[${context.label}] ${context.filename}:${line}:${column} — ${message}`
+ );
+}
+
+const FUNCTION_TYPES = ["FunctionExpression", "FunctionDeclaration"];
+
+function keyName(property) {
+ return property.key.name ?? property.key.value;
+}
+
+function isThisRouteCall(node) {
+ return (
+ node.type === "CallExpression" &&
+ node.callee.type === "MemberExpression" &&
+ !node.callee.computed &&
+ node.callee.object.type === "ThisExpression" &&
+ node.callee.property.name === "route"
+ );
+}
+
+function readOptions(context, node) {
+ const options = {};
+
+ for (const property of node.properties) {
+ if (property.type !== "Property" || property.computed) {
+ fail(context, property, "route options must be plain literal properties");
+ }
+
+ if (property.value.type !== "Literal") {
+ fail(
+ context,
+ property.value,
+ `option "${keyName(property)}" must be a literal`
+ );
+ }
+
+ options[keyName(property)] = property.value.value;
+ }
+
+ return options;
+}
+
+function readRouteCall(context, node) {
+ const [nameNode, ...rest] = node.arguments;
+
+ if (nameNode?.type !== "Literal" || typeof nameNode.value !== "string") {
+ fail(context, nameNode ?? node, "route name must be a string literal");
+ }
+
+ let options = {};
+ let children = [];
+
+ for (const argument of rest) {
+ if (argument.type === "ObjectExpression") {
+ options = readOptions(context, argument);
+ } else if (FUNCTION_TYPES.includes(argument.type)) {
+ children = readBody(context, argument.body);
+ } else if (argument.type === "ArrowFunctionExpression") {
+ // `this` inside an arrow is the enclosing scope, so the DSL cannot work there.
+ fail(context, argument, "route callback must be a function expression");
+ } else {
+ fail(context, argument, "unexpected argument to this.route");
+ }
+ }
+
+ return { name: nameNode.value, options, children };
+}
+
+function readBody(context, block) {
+ const routes = [];
+
+ for (const statement of block.body) {
+ if (
+ statement.type === "ExpressionStatement" &&
+ isThisRouteCall(statement.expression)
+ ) {
+ routes.push(readRouteCall(context, statement.expression));
+ continue;
+ }
+
+ // Core's maps build routes in loops. Those routes are missing from the derived tree.
+ if (context.lenient) {
+ continue;
+ }
+
+ fail(
+ context,
+ statement,
+ "a route map may only contain this.route calls with literal arguments"
+ );
+ }
+
+ return routes;
+}
+
+export function parseRouteMap(
+ ast,
+ { filename, source, label, lenient = false }
+) {
+ const context = { filename, source, label, lenient };
+
+ const exported = ast.body.find(
+ (node) => node.type === "ExportDefaultDeclaration"
+ );
+
+ if (!exported) {
+ fail(context, ast.body[0] ?? { start: 0 }, "no default export");
+ }
+
+ const declaration = exported.declaration;
+
+ if (FUNCTION_TYPES.includes(declaration.type)) {
+ return {
+ resource: null,
+ bundleName: null,
+ routes: readBody(context, declaration.body),
+ };
+ }
+
+ if (declaration.type !== "ObjectExpression") {
+ fail(
+ context,
+ declaration,
+ "a route map must export a function or an object"
+ );
+ }
+
+ const resource = declaration.properties.find(
+ (property) =>
+ property.type === "Property" && keyName(property) === "resource"
+ );
+ const map = declaration.properties.find(
+ (property) => property.type === "Property" && keyName(property) === "map"
+ );
+ const bundleName = declaration.properties.find(
+ (property) =>
+ property.type === "Property" && keyName(property) === "bundleName"
+ );
+
+ if (
+ bundleName &&
+ (bundleName.value.type !== "Literal" ||
+ typeof bundleName.value.value !== "string")
+ ) {
+ fail(context, bundleName.value, "`bundleName` must be a string literal");
+ }
+
+ if (!resource || resource.value.type !== "Literal") {
+ fail(
+ context,
+ resource ?? declaration,
+ "`resource` must be a string literal"
+ );
+ }
+
+ if (!map || !FUNCTION_TYPES.includes(map.value.type)) {
+ fail(context, map ?? declaration, "`map` must be a function");
+ }
+
+ // `mapping-router.js` reads only `resource` and `map`, so the rest is ours to use.
+ return {
+ resource: resource.value.value,
+ bundleName: bundleName?.value.value ?? null,
+ routes: readBody(context, map.value.body),
+ };
+}
+
+// Mirrors `RouteNode`: a child with the same name is merged into, so several maps can extend
+// one route.
+function addRoutes(parent, routes, core, bundleName = null) {
+ for (const route of routes) {
+ let node = parent.byName.get(route.name);
+
+ if (!node) {
+ const options = { ...route.options };
+
+ if (bundleName && options.bundleName === undefined) {
+ options.bundleName = bundleName;
+ }
+
+ node = {
+ name: route.name,
+ options,
+ core,
+ byName: new Map(),
+ };
+ parent.byName.set(route.name, node);
+ }
+
+ // Only the map's own top level is seeded, because everything below it inherits.
+ addRoutes(node, route.children, core);
+ }
+}
+
+// `resource` walks the names given to `this.route`, which are not route names:
+// `admin.adminPlugins.show` reaches a route named `adminPlugins.show`.
+function findPath(root, resource) {
+ let node = root;
+
+ for (const segment of resource.split(".")) {
+ node = node.byName.get(segment);
+
+ if (!node) {
+ return null;
+ }
+ }
+
+ return node;
+}
+
+export function buildRouteTree(maps) {
+ const root = { name: "root", options: {}, byName: new Map() };
+ const unmounted = [];
+
+ for (const map of maps.filter((candidate) => !candidate.resource)) {
+ addRoutes(root, map.routes, map.core, map.bundleName);
+ }
+
+ for (const map of maps.filter((candidate) => candidate.resource)) {
+ const node = findPath(root, map.resource);
+
+ if (node) {
+ addRoutes(node, map.routes, map.core, map.bundleName);
+ } else {
+ unmounted.push(map);
+ }
+ }
+
+ return { root, unmounted };
+}
+
+function joinUrl(parent, path) {
+ return [...parent.split("/"), ...path.split("/")].filter(Boolean).join("/");
+}
+
+// `Plugin::JsManager.url_glob_pattern` reads `*` as one segment and `**` as one or more.
+function globFor(url) {
+ return url
+ .split("/")
+ .map((segment) => {
+ if (segment.startsWith("*")) {
+ return "**";
+ }
+
+ return segment.startsWith(":") ? "*" : segment;
+ })
+ .join("/");
+}
+
+// Nothing a plugin declares is eager, so a route naming no bundle still falls into one.
+const DEFAULT_BUNDLE_NAME = "default";
+
+export function deriveRoutes(root) {
+ const derived = [];
+
+ function walk(nodes, parent) {
+ for (const node of nodes) {
+ const { options } = node;
+
+ const name =
+ options.resetNamespace || !parent.name
+ ? node.name
+ : `${parent.name}.${node.name}`;
+ const url = joinUrl(parent.url, options.path ?? node.name);
+ const bundleName =
+ options.bundleName ??
+ parent.bundleName ??
+ (node.core ? null : DEFAULT_BUNDLE_NAME);
+
+ derived.push({
+ name: dasherize(name),
+ url,
+ bundleName,
+ });
+
+ walk(node.byName.values(), { name, url, bundleName });
+ }
+ }
+
+ walk(root.byName.values(), { name: "", url: "", bundleName: null });
+
+ return derived;
+}
+
+function literalSegments(glob) {
+ return glob.split("/").filter((segment) => !/^\*\*?$/.test(segment)).length;
+}
+
+function segmentCount(glob) {
+ return glob.split("/").length;
+}
+
+// A glob a shallower one in the same bundle already covers adds nothing.
+function urlsFor(globs) {
+ const sorted = [...new Set(globs)].sort();
+
+ return sorted.filter(
+ (glob) =>
+ !sorted.some((other) => other !== glob && glob.startsWith(`${other}/`))
+ );
+}
+
+export function bundleByRouteFor(derived) {
+ const bundleByRoute = {};
+
+ for (const route of derived) {
+ if (route.bundleName) {
+ bundleByRoute[route.name] = route.bundleName;
+ }
+ }
+
+ return bundleByRoute;
+}
+
+// Urls come out most specific first, so the first match wins whatever order routes were declared.
+export function urlTableFor(derived) {
+ const globsByBundle = new Map();
+
+ for (const route of derived) {
+ if (!route.bundleName) {
+ continue;
+ }
+
+ const globs = globsByBundle.get(route.bundleName) ?? [];
+ globs.push(globFor(route.url));
+ globsByBundle.set(route.bundleName, globs);
+ }
+
+ return [...globsByBundle]
+ .flatMap(([bundleName, globs]) =>
+ urlsFor(globs).map((glob) => ({ bundleName, url: `${glob}/*` }))
+ )
+ .sort(
+ (a, b) =>
+ literalSegments(b.url) - literalSegments(a.url) ||
+ segmentCount(b.url) - segmentCount(a.url)
+ );
+}
diff --git a/frontend/asset-processor/route-map-parser.test.mjs b/frontend/asset-processor/route-map-parser.test.mjs
new file mode 100644
index 00000000000..0eb0b139e9f
--- /dev/null
+++ b/frontend/asset-processor/route-map-parser.test.mjs
@@ -0,0 +1,299 @@
+import { parseAst as parse } from "rolldown/parseAst";
+import { describe, expect, it } from "vitest";
+import {
+ buildRouteTree,
+ bundleByRouteFor,
+ deriveRoutes,
+ parseRouteMap,
+ urlTableFor,
+} from "./route-map-parser";
+
+function read(source, opts = {}) {
+ return parseRouteMap(parse(source), {
+ filename: "test-route-map.js",
+ source,
+ label: "PLUGIN test",
+ ...opts,
+ });
+}
+
+// `core: true` stands in for core's own maps, whose routes get no implicit bundle.
+function tablesFor(sources) {
+ const maps = sources.map((source) =>
+ typeof source === "string"
+ ? read(source)
+ : { core: source.core, ...read(source.source, source) }
+ );
+ const derived = deriveRoutes(buildRouteTree(maps).root);
+
+ return {
+ bundleByRoute: bundleByRouteFor(derived),
+ urls: urlTableFor(derived),
+ };
+}
+
+describe("parseRouteMap", () => {
+ it("reads a function map", () => {
+ const map = read(`export default function () {
+ this.route("chat", { bundleName: "chat" }, function () {
+ this.route("channel", { path: "/c/:channelTitle/:channelId" });
+ });
+ }`);
+
+ expect(map.resource).toBe(null);
+ expect(map.routes[0].name).toBe("chat");
+ expect(map.routes[0].options).toEqual({ bundleName: "chat" });
+ expect(map.routes[0].children[0].options).toEqual({
+ path: "/c/:channelTitle/:channelId",
+ });
+ });
+
+ it("reads a resource map, ignoring `path` the router ignores", () => {
+ const map = read(`export default {
+ resource: "admin.adminPlugins.show",
+ path: "/plugins",
+ map() {
+ this.route("hooks");
+ },
+ };`);
+
+ expect(map.resource).toBe("admin.adminPlugins.show");
+ expect(map.routes).toHaveLength(1);
+ });
+
+ it.each([
+ [
+ "a loop",
+ `export default function () { for (const x of []) { this.route(x); } }`,
+ ],
+ [
+ "a conditional",
+ `export default function () { if (window.x) { this.route("a"); } }`,
+ ],
+ ["a computed name", `export default function () { this.route(NAME); }`],
+ [
+ "a spread",
+ `export default function () { this.route("a", { ...opts }); }`,
+ ],
+ [
+ "a non-literal option",
+ `export default function () { this.route("a", { path: X }); }`,
+ ],
+ ["another call", `export default function () { setup(); }`],
+ [
+ "an arrow callback",
+ `export default function () { this.route("a", () => {}); }`,
+ ],
+ ["no default export", `export function map() {}`],
+ [
+ "a `map` reference",
+ `export default { resource: "a", map: someFunction };`,
+ ],
+ ])("rejects %s", (_label, source) => {
+ expect(() => read(source)).toThrow(/test-route-map\.js:\d+:\d+/);
+ });
+
+ it("skips what it cannot read when lenient", () => {
+ const map = read(
+ `export default function () {
+ list.forEach((x) => this.route(x));
+ this.route("about");
+ }`,
+ { lenient: true }
+ );
+
+ expect(map.routes.map((route) => route.name)).toEqual(["about"]);
+ });
+});
+
+describe("deriving", () => {
+ it("nests names and paths, and resets only the name", () => {
+ const { urls, bundleByRoute } = tablesFor([
+ {
+ core: true,
+ source: `export default function () {
+ this.route("admin", function () {
+ this.route("adminPlugins", { path: "/plugins", resetNamespace: true }, function () {
+ this.route("show", { path: "/:plugin_id" });
+ });
+ });
+ }`,
+ },
+ `export default {
+ resource: "admin.adminPlugins.show",
+ map() {
+ this.route("discourse-chat-incoming-webhooks", { path: "hooks", bundleName: "webhooks" }, function () {
+ this.route("edit", { path: "/:id/edit" });
+ });
+ },
+ };`,
+ ]);
+
+ expect(bundleByRoute).toEqual({
+ "admin-plugins.show.discourse-chat-incoming-webhooks": "webhooks",
+ "admin-plugins.show.discourse-chat-incoming-webhooks.edit": "webhooks",
+ });
+ expect(urls).toEqual([
+ { bundleName: "webhooks", url: "admin/plugins/*/hooks/*" },
+ ]);
+ });
+
+ it("gives a plugin route with no bundleName the default bundle", () => {
+ const { bundleByRoute, urls } = tablesFor([
+ {
+ core: true,
+ source: `export default function () { this.route("user"); }`,
+ },
+ `export default function () {
+ this.route("standalone");
+ this.route("named", { bundleName: "named" });
+ }`,
+ ]);
+
+ expect(bundleByRoute).toEqual({ standalone: "default", named: "named" });
+ expect(bundleByRoute.user).toBeUndefined();
+ expect(urls).toContainEqual({ bundleName: "default", url: "standalone/*" });
+ });
+
+ it("groups siblings into one bundle", () => {
+ const { bundleByRoute, urls } = tablesFor([
+ `export default function () {
+ this.route("chat", { bundleName: "chat" }, function () {
+ this.route("disabled", { bundleName: "foo" });
+ this.route("search", { bundleName: "foo" });
+ });
+ }`,
+ ]);
+
+ expect(bundleByRoute).toEqual({
+ chat: "chat",
+ "chat.disabled": "foo",
+ "chat.search": "foo",
+ });
+ expect(urls).toEqual([
+ { bundleName: "foo", url: "chat/disabled/*" },
+ { bundleName: "foo", url: "chat/search/*" },
+ { bundleName: "chat", url: "chat/*" },
+ ]);
+ });
+
+ it("keeps a splat distinct from a dynamic segment", () => {
+ // A `:dynamic` is one segment, a `*splat` is the rest of the route path.
+ const { urls } = tablesFor([
+ {
+ core: true,
+ source: `export default function () {
+ this.route("discovery", { path: "/" }, function () {
+ this.route("category", { path: "/c/*category_slug_path_with_id" });
+ });
+ }`,
+ },
+ `export default {
+ resource: "discovery.category",
+ map() {
+ this.route("extras", { path: "/:id/extras", bundleName: "extras" });
+ },
+ };`,
+ ]);
+
+ expect(urls).toEqual([{ bundleName: "extras", url: "c/**/*/extras/*" }]);
+ });
+
+ it("orders urls most specific first", () => {
+ const { urls } = tablesFor([
+ `export default function () {
+ this.route("chat", { bundleName: "chat" }, function () {
+ this.route("search", { bundleName: "foo" });
+ });
+ }`,
+ ]);
+
+ expect(urls.map((entry) => entry.url)).toEqual(["chat/search/*", "chat/*"]);
+ });
+
+ it("takes a bundle name from the map itself", () => {
+ const { bundleByRoute } = tablesFor([
+ `export default {
+ resource: "admin",
+ bundleName: "workflows",
+ map() {
+ this.route("workflows", function () { this.route("show"); });
+ this.route("variables");
+ },
+ };`,
+ `export default function () { this.route("admin"); }`,
+ ]);
+
+ expect(bundleByRoute["admin.workflows"]).toBe("workflows");
+ expect(bundleByRoute["admin.workflows.show"]).toBe("workflows");
+ expect(bundleByRoute["admin.variables"]).toBe("workflows");
+ });
+
+ it("lets a route override the bundle the map names", () => {
+ const { bundleByRoute } = tablesFor([
+ `export default {
+ resource: "admin",
+ bundleName: "workflows",
+ map() {
+ this.route("a");
+ this.route("b", { bundleName: "other" }, function () {
+ this.route("c");
+ });
+ },
+ };`,
+ `export default function () { this.route("admin"); }`,
+ ]);
+
+ expect(bundleByRoute["admin.a"]).toBe("workflows");
+ expect(bundleByRoute["admin.b"]).toBe("other");
+ expect(bundleByRoute["admin.b.c"]).toBe("other");
+ });
+
+ it("rejects a computed bundle name on the map", () => {
+ expect(() =>
+ tablesFor([
+ `export default {
+ resource: "admin",
+ bundleName: SOME_CONSTANT,
+ map() { this.route("a"); },
+ };`,
+ ])
+ ).toThrow(/`bundleName` must be a string literal/);
+ });
+
+ it("orders a dynamic child ahead of the parent it sits under", () => {
+ const { urls } = tablesFor([
+ `export default function () {
+ this.route("workflows", { bundleName: "workflows" }, function () {
+ this.route("show", { path: "/:id", bundleName: "editor" });
+ });
+ }`,
+ ]);
+
+ expect(urls.map((entry) => entry.url)).toEqual([
+ "workflows/*/*",
+ "workflows/*",
+ ]);
+ });
+
+ it("merges maps that extend the same route", () => {
+ const { bundleByRoute } = tablesFor([
+ `export default function () { this.route("chat", { bundleName: "chat" }); }`,
+ `export default function () {
+ this.route("chat", function () { this.route("extra"); });
+ }`,
+ ]);
+
+ expect(bundleByRoute["chat.extra"]).toBe("chat");
+ });
+
+ it("reports a resource that mounts on nothing", () => {
+ const { unmounted } = buildRouteTree([
+ read(
+ `export default { resource: "discovery.filter", map() { this.route("x"); } };`
+ ),
+ ]);
+
+ expect(unmounted).toHaveLength(1);
+ });
+});
diff --git a/frontend/discourse/app/app.js b/frontend/discourse/app/app.js
index c0ef17a0a2f..2aa1817b795 100644
--- a/frontend/discourse/app/app.js
+++ b/frontend/discourse/app/app.js
@@ -61,13 +61,37 @@ window.moduleBroker = {
},
};
+// `Resolver#addModules` expects the same namespacing as the eager modules.
+function registerRouteBundles(bundle, prefix) {
+ window._embroiderRouteBundles_ ??= [];
+
+ for (const { names, load } of bundle.routes ?? []) {
+ window._embroiderRouteBundles_.push({
+ names,
+ load: async () => {
+ const routeModules = (await load()).default;
+
+ return {
+ default: Object.fromEntries(
+ Object.entries(routeModules).map(([key, mod]) => [
+ `${prefix}/${key}`,
+ mod,
+ ])
+ ),
+ };
+ },
+ });
+ }
+}
+
async function loadThemeFromModulePreload(link) {
const themeId = link.dataset.themeId;
try {
- const compatModules = (await import(/* @vite-ignore */ link.href)).default;
- for (const [key, mod] of Object.entries(compatModules)) {
+ const bundle = await import(/* @vite-ignore */ link.href);
+ for (const [key, mod] of Object.entries(bundle.compatModules)) {
define(`discourse/theme-${themeId}/${key}`, () => mod);
}
+ registerRouteBundles(bundle, `discourse/theme-${themeId}`);
} catch (error) {
// eslint-disable-next-line no-console
console.error(
@@ -86,10 +110,11 @@ async function loadThemeFromModulePreload(link) {
async function loadPluginFromModulePreload(link) {
const pluginName = link.dataset.pluginName;
try {
- const compatModules = (await import(/* @vite-ignore */ link.href)).default;
- for (const [key, mod] of Object.entries(compatModules)) {
+ const bundle = await import(/* @vite-ignore */ link.href);
+ for (const [key, mod] of Object.entries(bundle.compatModules)) {
define(`discourse/plugins/${pluginName}/${key}`, () => mod);
}
+ registerRouteBundles(bundle, `discourse/plugins/${pluginName}`);
} catch (error) {
if (DEBUG) {
if (isRailsTesting() || isTesting()) {
diff --git a/frontend/discourse/app/lib/deferred-class-modifications.js b/frontend/discourse/app/lib/deferred-class-modifications.js
new file mode 100644
index 00000000000..16388304b68
--- /dev/null
+++ b/frontend/discourse/app/lib/deferred-class-modifications.js
@@ -0,0 +1,52 @@
+// A `modifyClass` call can name a class whose module has not loaded yet. Route bundles flush
+// these when they arrive; a component registers itself, because nothing else registers it.
+
+const pendingModifications = new Map();
+const lazyClasses = new Map();
+
+export function lazyClassFor(resolverName) {
+ const klass = lazyClasses.get(resolverName);
+
+ return klass ? { class: klass } : null;
+}
+
+export function deferClassModification(resolverName, pendingModification) {
+ const modifications = pendingModifications.get(resolverName) ?? [];
+
+ modifications.push(pendingModification);
+ pendingModifications.set(resolverName, modifications);
+}
+
+// Appended to every plugin and theme component module by `discourse-register-components`.
+export function registerModuleForModifyClass(path, klass) {
+ const resolverName = `component:${path}`;
+
+ lazyClasses.set(resolverName, klass);
+ applyPending(resolverName);
+}
+
+export function resetDeferredClassModifications() {
+ pendingModifications.clear();
+}
+
+// Route modules arrive as a batch through `Resolver#addModules` rather than one at a time.
+export function applyDeferredClassModifications() {
+ for (const resolverName of [...pendingModifications.keys()]) {
+ applyPending(resolverName);
+ }
+}
+
+// Taking them off first is what stops one that defers again from looping.
+function applyPending(resolverName) {
+ const modifications = pendingModifications.get(resolverName);
+
+ if (!modifications) {
+ return;
+ }
+
+ pendingModifications.delete(resolverName);
+
+ for (const pendingModification of modifications) {
+ pendingModification();
+ }
+}
diff --git a/frontend/discourse/app/lib/plugin-api.gjs b/frontend/discourse/app/lib/plugin-api.gjs
index 6292957b2bb..428db530ebd 100644
--- a/frontend/discourse/app/lib/plugin-api.gjs
+++ b/frontend/discourse/app/lib/plugin-api.gjs
@@ -67,6 +67,10 @@ import {
_INTERNAL_SOURCE_KEY,
CORE_SOURCE,
} from "discourse/lib/customization-source";
+import {
+ deferClassModification,
+ lazyClassFor,
+} from "discourse/lib/deferred-class-modifications";
import deprecated from "discourse/lib/deprecated";
import { registerDesktopNotificationHandler } from "discourse/lib/desktop-notifications";
import { downloadCalendar } from "discourse/lib/download-calendar";
@@ -238,8 +242,7 @@ class _PluginApi {
return this.container.lookup(path);
}
- _resolveClass(resolverName, opts) {
- opts = opts || {};
+ _resolveClass(resolverName) {
const normalized = this.container.registry.normalize(resolverName);
if (
this.container.cache[normalized] ||
@@ -256,18 +259,7 @@ class _PluginApi {
let klass;
if (!blockedModifications.includes(normalized)) {
- klass = this.container.factoryFor(normalized);
- }
-
- if (!klass) {
- if (!opts.ignoreMissing) {
- // eslint-disable-next-line no-console
- console.warn(
- consolePrefix(),
- `"${normalized}" was not found by modifyClass`
- );
- }
- return;
+ klass = this.container.factoryFor(normalized) || lazyClassFor(normalized);
}
return klass;
@@ -295,8 +287,11 @@ class _PluginApi {
modifyClass(resolverName, changes, opts) {
this.#deprecateModifyClass(resolverName, "modifyClass");
- const klass = this._resolveClass(resolverName, opts);
+ const klass = this._resolveClass(resolverName);
if (!klass) {
+ this.#deferModification(resolverName, () =>
+ this.modifyClass(resolverName, changes, opts)
+ );
return;
}
@@ -336,8 +331,11 @@ class _PluginApi {
modifyClassStatic(resolverName, changes, opts) {
this.#deprecateModifyClass(resolverName, "modifyClassStatic");
- const klass = this._resolveClass(resolverName, opts);
+ const klass = this._resolveClass(resolverName);
if (!klass) {
+ this.#deferModification(resolverName, () =>
+ this.modifyClassStatic(resolverName, changes, opts)
+ );
return;
}
@@ -3943,6 +3941,14 @@ class _PluginApi {
_registerConditionType(ConditionClass, this.source);
}
+ // The module may not have been evaluated yet, and registers itself when it is.
+ #deferModification(resolverName, retry) {
+ deferClassModification(
+ this.container.registry.normalize(resolverName),
+ retry
+ );
+ }
+
#deprecateModifyClass(resolverName, apiName) {
if (!resolverName.startsWith("model:")) {
return;
diff --git a/frontend/discourse/app/loader-shims.js b/frontend/discourse/app/loader-shims.js
index 625742993dc..4b6caae8787 100644
--- a/frontend/discourse/app/loader-shims.js
+++ b/frontend/discourse/app/loader-shims.js
@@ -63,6 +63,8 @@ loaderShim("@ember/template-factory", () =>
importSync("@ember/template-factory")
);
loaderShim("@ember/template", () => importSync("@ember/template"));
+// Needed in production: plugins register their dynamic imports so tests wait for them.
+loaderShim("@ember/test-waiters", () => importSync("@ember/test-waiters"));
loaderShim("@ember/utils", () => importSync("@ember/utils"));
loaderShim("@floating-ui/dom", () => importSync("@floating-ui/dom"));
loaderShim("@glimmer/component", () => importSync("@glimmer/component"));
diff --git a/frontend/discourse/app/resolver.js b/frontend/discourse/app/resolver.js
index 670a22e72be..e5917e8d5f1 100644
--- a/frontend/discourse/app/resolver.js
+++ b/frontend/discourse/app/resolver.js
@@ -1,5 +1,6 @@
import { dasherize, decamelize } from "@ember/string";
import Resolver from "ember-resolver";
+import { applyDeferredClassModifications } from "discourse/lib/deferred-class-modifications";
import deprecated, { withSilencedDeprecations } from "discourse/lib/deprecated";
import DiscourseTemplateMap from "discourse/lib/discourse-template-map";
import { findHelper } from "discourse/lib/helpers";
@@ -371,6 +372,12 @@ export function buildResolver(baseName) {
for (const [key, value] of Object.entries(modules)) {
define(key, () => value);
}
+
+ // Both are memoized, so a lazily-loaded route bundle is invisible to them until rebuilt.
+ expireModuleTrieCache();
+ DiscourseTemplateMap.setModuleNames(Object.keys(requirejs.entries));
+
+ applyDeferredClassModifications();
}
/**
diff --git a/frontend/discourse/tests/helpers/qunit-helpers.js b/frontend/discourse/tests/helpers/qunit-helpers.js
index 4fb2425ddce..f80ee04a8d2 100644
--- a/frontend/discourse/tests/helpers/qunit-helpers.js
+++ b/frontend/discourse/tests/helpers/qunit-helpers.js
@@ -48,6 +48,7 @@ import { resetAdditionalReportModes } from "discourse/lib/admin-report-additiona
import { rollbackAllPrepends } from "discourse/lib/class-prepend";
import { _clearRegisteredActions } from "discourse/lib/composer/actions-registry";
import { clearPopupMenuOptions } from "discourse/lib/composer/custom-popup-menu-options";
+import { resetDeferredClassModifications } from "discourse/lib/deferred-class-modifications";
import deprecated, { clearBacklog } from "discourse/lib/deprecated";
import { clearDesktopNotificationHandlers } from "discourse/lib/desktop-notifications";
import { visible as isVisible } from "discourse/lib/dom-utils";
@@ -224,6 +225,7 @@ export function testCleanup(container, app) {
resetAdminDashboardReportRenderers();
resetAdminReportRelatedItemsRenderers();
resetAdminDashboardSections();
+ resetDeferredClassModifications();
resetExtraClasses();
clearOutletCache();
clearHTMLCache();
diff --git a/frontend/discourse/tests/unit/ember/resolver-test.js b/frontend/discourse/tests/unit/ember/resolver-test.js
index 6e2405dc3f4..ae0202e60c5 100644
--- a/frontend/discourse/tests/unit/ember/resolver-test.js
+++ b/frontend/discourse/tests/unit/ember/resolver-test.js
@@ -1,8 +1,12 @@
import { setupTest } from "ember-qunit";
import { module, test } from "qunit";
+import {
+ applyDeferredClassModifications,
+ deferClassModification,
+} from "discourse/lib/deferred-class-modifications";
import { withSilencedDeprecations } from "discourse/lib/deprecated";
import DiscourseTemplateMap from "discourse/lib/discourse-template-map";
-import { buildResolver } from "discourse/resolver";
+import { buildResolver, expireModuleTrieCache } from "discourse/resolver";
import { registerTemporaryModule } from "discourse/tests/helpers/temporary-module-helper";
let resolver;
@@ -632,4 +636,81 @@ module("Unit | Ember | resolver", function (hooks) {
"my-second-component"
);
});
+
+ test("addModules exposes modules which arrive after boot", function (assert) {
+ const templateModule =
+ "discourse/plugins/my-fake-plugin/discourse/templates/lazy-route";
+ const serviceModule =
+ "discourse/plugins/my-fake-plugin/discourse/services/lazy-thing";
+
+ // Build both caches first, the way a lazily-loaded route bundle finds them.
+ expireModuleTrieCache();
+ DiscourseTemplateMap.setModuleNames(Object.keys(requirejs.entries));
+ resolve("service:some-service-to-populate-the-trie");
+
+ try {
+ resolver.addModules({
+ [templateModule]: { default: "lazy-template" },
+ [serviceModule]: { default: "lazy-service" },
+ });
+
+ lookupTemplate(
+ assert,
+ "template:lazy-route",
+ "lazy-template",
+ "finds a template added after the template map was built"
+ );
+ assert.strictEqual(
+ resolve("service:lazy-thing"),
+ "lazy-service",
+ "finds a service added after the suffix trie was built"
+ );
+ } finally {
+ delete requirejs.entries[templateModule];
+ delete requirejs.entries[serviceModule];
+ expireModuleTrieCache();
+ DiscourseTemplateMap.setModuleNames(Object.keys(requirejs.entries));
+ }
+ });
+
+ test("addModules applies modifications deferred for a route bundle", function (assert) {
+ const routeModule =
+ "discourse/plugins/my-fake-plugin/discourse/routes/lazy-route";
+ const controllerModule =
+ "discourse/plugins/my-fake-plugin/discourse/controllers/lazy-route";
+
+ expireModuleTrieCache();
+ resolve("service:some-service-to-populate-the-trie");
+
+ let applied = [];
+ deferClassModification("route:lazy-route", () => applied.push("route"));
+ deferClassModification("controller:lazy-route", () =>
+ applied.push("controller")
+ );
+
+ assert.deepEqual(applied, [], "waits for the bundle");
+
+ try {
+ resolver.addModules({
+ [routeModule]: { default: "lazy-route-class" },
+ [controllerModule]: { default: "lazy-controller-class" },
+ });
+
+ assert.deepEqual(
+ applied.sort(),
+ ["controller", "route"],
+ "applies both once the bundle arrives"
+ );
+ assert.strictEqual(resolve("route:lazy-route"), "lazy-route-class");
+ assert.strictEqual(
+ resolve("controller:lazy-route"),
+ "lazy-controller-class"
+ );
+ } finally {
+ delete requirejs.entries[routeModule];
+ delete requirejs.entries[controllerModule];
+ expireModuleTrieCache();
+ applyDeferredClassModifications();
+ }
+ });
});
diff --git a/frontend/discourse/tests/unit/lib/deferred-class-modifications-test.js b/frontend/discourse/tests/unit/lib/deferred-class-modifications-test.js
new file mode 100644
index 00000000000..5d74176e349
--- /dev/null
+++ b/frontend/discourse/tests/unit/lib/deferred-class-modifications-test.js
@@ -0,0 +1,60 @@
+import { setupTest } from "ember-qunit";
+import { module, test } from "qunit";
+import {
+ applyDeferredClassModifications,
+ deferClassModification,
+ lazyClassFor,
+ registerModuleForModifyClass,
+ resetDeferredClassModifications,
+} from "discourse/lib/deferred-class-modifications";
+
+module("Unit | Lib | deferred-class-modifications", function (hooks) {
+ setupTest(hooks);
+
+ hooks.afterEach(function () {
+ resetDeferredClassModifications();
+ });
+
+ test("keys a component by the name the resolver asks for", function (assert) {
+ class Icon {}
+ registerModuleForModifyClass("chat/header/icon", Icon);
+
+ assert.strictEqual(lazyClassFor("component:chat/header/icon").class, Icon);
+ assert.strictEqual(lazyClassFor("component:missing"), null);
+ });
+
+ test("applies a modification that arrived before the module", function (assert) {
+ class Header {}
+ let applied = 0;
+
+ deferClassModification("component:chat-header", () => applied++);
+ assert.strictEqual(applied, 0, "waits for the module");
+
+ registerModuleForModifyClass("chat-header", Header);
+ assert.strictEqual(applied, 1);
+
+ registerModuleForModifyClass("chat-header", Header);
+ assert.strictEqual(applied, 1);
+ });
+
+ test("drops a modification whose module never arrives", function (assert) {
+ let applied = 0;
+ deferClassModification("component:never-loaded", () => applied++);
+
+ resetDeferredClassModifications();
+ applyDeferredClassModifications();
+
+ assert.strictEqual(applied, 0);
+ });
+
+ test("leaves modifications for other names alone", function (assert) {
+ let applied = 0;
+ deferClassModification("component:not-this-one", () => applied++);
+
+ registerModuleForModifyClass("chat-header", class {});
+ assert.strictEqual(applied, 0);
+
+ applyDeferredClassModifications();
+ assert.strictEqual(applied, 1);
+ });
+});
diff --git a/lib/asset_processor.rb b/lib/asset_processor.rb
index 9dfe17f9ee4..065495e3a35 100644
--- a/lib/asset_processor.rb
+++ b/lib/asset_processor.rb
@@ -1,7 +1,7 @@
# frozen_string_literal: true
class AssetProcessor
- BASE_COMPILER_VERSION = 114
+ BASE_COMPILER_VERSION = 119
BUNDLE =
PrecompiledBundle.new(
diff --git a/lib/discourse.rb b/lib/discourse.rb
index c9ad6a8aba3..c0387760b17 100644
--- a/lib/discourse.rb
+++ b/lib/discourse.rb
@@ -495,6 +495,8 @@ module Discourse
plugins = apply_asset_filters(plugins, :js, args[:request])
+ request_path = args[:request]&.path&.delete_prefix(Discourse.base_path)&.delete_prefix("/")
+
assets = []
plugins.each do |plugin|
@@ -508,6 +510,8 @@ module Discourse
importmap_name: "discourse/plugins/#{plugin.name}",
external_plugin_imports:
Plugin::JsManager.external_plugin_imports(plugin.directory_name, "main"),
+ route_bundle:
+ Plugin::JsManager.route_bundle_for_path(plugin.directory_name, "main", request_path),
}
end
end
@@ -530,6 +534,8 @@ module Discourse
type_module: true,
external_plugin_imports:
Plugin::JsManager.external_plugin_imports(plugin.directory_name, "admin"),
+ route_bundle:
+ Plugin::JsManager.route_bundle_for_path(plugin.directory_name, "admin", request_path),
}
end
end
diff --git a/lib/plugin/js_compiler.rb b/lib/plugin/js_compiler.rb
index 5c15dbc399a..07464899f21 100644
--- a/lib/plugin/js_compiler.rb
+++ b/lib/plugin/js_compiler.rb
@@ -7,7 +7,8 @@ class Plugin::JsCompiler
tree: {},
entrypoints: {},
filename_prefix: nil,
- filename_suffix: nil
+ filename_suffix: nil,
+ frontend_config: nil
)
@plugin_name = plugin_name
@tree = tree
@@ -15,6 +16,7 @@ class Plugin::JsCompiler
@minify = minify
@filename_prefix = filename_prefix
@filename_suffix = filename_suffix
+ @frontend_config = frontend_config
end
def compile!
@@ -26,6 +28,7 @@ class Plugin::JsCompiler
entrypoints: @entrypoints,
filenamePrefix: @filename_prefix,
filenameSuffix: @filename_suffix,
+ frontendConfig: @frontend_config,
},
)
rescue AssetProcessor::TimeoutError => e
diff --git a/lib/plugin/js_manager.rb b/lib/plugin/js_manager.rb
index 0ab91a992df..e3c432db84c 100644
--- a/lib/plugin/js_manager.rb
+++ b/lib/plugin/js_manager.rb
@@ -7,6 +7,14 @@ module Plugin
attribute :request_cache
end
+ @url_glob_patterns = {}
+
+ # A plugin route map can mount on a core route, so the build needs these to derive its urls.
+ CORE_ROUTE_MAPS = {
+ "app-route-map.js" => "frontend/discourse/app/routes/app-route-map.js",
+ "admin-route-map.js" => "frontend/discourse/admin/routes/admin-route-map.js",
+ }.freeze
+
def self.optional_plugin_stub
"data:text/javascript,/* autogenerated missing optional plugin stub */const m=new Proxy({},{get:()=>null});export default new Proxy({},{get:()=>m});"
end
@@ -47,21 +55,60 @@ module Plugin
end
end
+ def self.logical_path(file_name)
+ "js/plugins/#{file_name.delete_suffix(".js")}"
+ end
+
def self.digested_logical_path_for(plugin_directory_name, entrypoint_name)
manifest_entry = read_manifest(plugin_directory_name)[entrypoint_name]
- "js/plugins/#{manifest_entry["fileName"].delete_suffix(".js")}" if manifest_entry
+ logical_path(manifest_entry["fileName"]) if manifest_entry
end
def self.import_paths_for(plugin_directory_name, entrypoint_name)
- read_manifest(plugin_directory_name)[entrypoint_name]["imports"].map do
- "js/plugins/#{it.delete_suffix(".js")}"
- end
+ read_manifest(plugin_directory_name)[entrypoint_name]["imports"].map { logical_path(it) }
end
def self.external_plugin_imports(plugin_directory_name, entrypoint_name)
read_manifest(plugin_directory_name)[entrypoint_name]["externalPluginImports"]
end
+ # The first match wins, so the build emits the most specific glob first.
+ def self.route_bundle_for_path(plugin_directory_name, entrypoint_name, path)
+ return if path.nil?
+
+ bundles = read_manifest(plugin_directory_name).dig(entrypoint_name, "routeBundles")
+
+ bundle = bundles.to_a.find { |c| path.match?(url_glob_pattern(c["url"])) }
+
+ logical_path(bundle["fileName"]) if bundle
+ end
+
+ # Every url the build emits ends in `/*`, so a bundle covers its own route and everything
+ # beneath it. Within the rest, `*` is one segment and `**` is one or more, standing in for a
+ # splat route segment. A pattern only depends on its glob, so it never goes stale.
+ def self.url_glob_pattern(glob)
+ @url_glob_patterns[glob] ||= begin
+ pattern =
+ glob
+ .delete_suffix("/*")
+ .split("/")
+ .map do |segment|
+ case segment
+ when "**"
+ "[^/]+(?:/[^/]+)*"
+ when "*"
+ "[^/]+"
+ else
+ Regexp.escape(segment)
+ end
+ end
+ .join("/")
+
+ %r{\A#{pattern}(?:/.*)?\z}
+ end
+ end
+ private_class_method :url_glob_pattern
+
def compile!
log "Compiling #{Discourse.plugins.count} plugins..."
start = Time.now
@@ -120,6 +167,17 @@ module Plugin
end
end
+ frontend_config = plugin.about_json_metadata&.dig("frontend")
+
+ # Only a plugin that splits its own routes derives anything from these. Read from source, so
+ # plugin compilation does not wait on core's build. Being in `tree` puts them in the digest
+ # below, so a core route change rebuilds those plugins and no others.
+ if frontend_config&.dig("staticModules") && tree.keys.any? { it.end_with?("route-map.js") }
+ CORE_ROUTE_MAPS.each do |name, source_path|
+ tree["__core__/#{name}"] = File.read(Rails.root.join(source_path))
+ end
+ end
+
hex_digest =
Digest::SHA1.hexdigest(
[
@@ -129,6 +187,7 @@ module Plugin
AssetProcessor.ember_version,
minify?.to_s,
plugin.name,
+ frontend_config.to_json,
].join,
)
base36_digest = hex_digest.to_i(16).to_s(36).first(8)
@@ -155,6 +214,7 @@ module Plugin
entrypoints: entrypoints_config,
filename_prefix:,
filename_suffix:,
+ frontend_config:,
)
result = compiler.compile!
@@ -174,6 +234,7 @@ module Plugin
fileName: file_name,
imports: info["imports"],
externalPluginImports: info["externalPluginImports"],
+ routeBundles: info["routeBundles"],
}
end
end
diff --git a/spec/lib/asset_processor_spec.rb b/spec/lib/asset_processor_spec.rb
index a1a1428dcf6..74614d9aea9 100644
--- a/spec/lib/asset_processor_spec.rb
+++ b/spec/lib/asset_processor_spec.rb
@@ -478,8 +478,9 @@ RSpec.describe AssetProcessor do
expect(entrypoint(result, "main")["code"]).to include("setComponentTemplate")
expect(entrypoint(result, "main")["code"]).to include(
- "bar = setComponentTemplate(__COLOCATED_TEMPLATE__, templateOnly());",
+ "= setComponentTemplate(__COLOCATED_TEMPLATE__, templateOnly());",
)
+ expect(entrypoint(result, "main")["code"]).to include('registerModuleForModifyClass("bar",')
end
it "handles colocation of connectors" do
diff --git a/spec/lib/plugin/js_manager_spec.rb b/spec/lib/plugin/js_manager_spec.rb
new file mode 100644
index 00000000000..ddb726fcdb8
--- /dev/null
+++ b/spec/lib/plugin/js_manager_spec.rb
@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+RSpec.describe Plugin::JsManager do
+ describe ".route_bundle_for_path" do
+ let(:manifest) do
+ {
+ "main" => {
+ "routeBundles" => [
+ { "url" => "chat/visualizer/*", "fileName" => "visualizer.js" },
+ { "url" => "chat/*", "fileName" => "chat.js" },
+ { "url" => "u/*/preferences/chat/*", "fileName" => "preferences.js" },
+ ],
+ },
+ }
+ end
+
+ before { allow(described_class).to receive(:read_manifest).and_return(manifest) }
+
+ def bundle_for(path)
+ described_class.route_bundle_for_path("chat", "main", path)
+ end
+
+ it "matches an exact url" do
+ expect(bundle_for("chat/visualizer")).to eq("js/plugins/visualizer")
+ end
+
+ it "prefers the first matching glob, so a specific url wins over a wildcard covering it" do
+ # `chat/visualizer` also matches `chat/*`, but is declared first.
+ expect(bundle_for("chat/visualizer")).to eq("js/plugins/visualizer")
+ expect(bundle_for("chat/c/some-channel/1")).to eq("js/plugins/chat")
+ end
+
+ it "matches the bare parent of a trailing wildcard" do
+ expect(bundle_for("chat")).to eq("js/plugins/chat")
+ end
+
+ it "does not let a wildcard match a sibling with the same prefix" do
+ expect(bundle_for("chatter")).to eq(nil)
+ end
+
+ it "matches a wildcard standing in for a dynamic segment" do
+ expect(bundle_for("u/bob/preferences/chat")).to eq("js/plugins/preferences")
+ end
+
+ it "requires the segments after a wildcard to match" do
+ expect(bundle_for("u/bob/preferences/account")).to eq(nil)
+ expect(bundle_for("u/bob/preferences")).to eq(nil)
+ end
+
+ context "with a splat segment" do
+ let(:manifest) do
+ { "main" => { "routeBundles" => [{ "url" => "c/**/edit/*", "fileName" => "edit.js" }] } }
+ end
+
+ it "matches one segment or many, because a splat eats the rest of the route path" do
+ expect(bundle_for("c/announcements/edit")).to eq("js/plugins/edit")
+ expect(bundle_for("c/parent/child/5/edit")).to eq("js/plugins/edit")
+ end
+
+ it "still requires the segments around it to match" do
+ expect(bundle_for("c/announcements/5")).to eq(nil)
+ expect(bundle_for("c/edit")).to eq(nil)
+ expect(bundle_for("x/announcements/5/edit")).to eq(nil)
+ end
+ end
+
+ it "returns nothing for an unrelated path" do
+ expect(bundle_for("latest")).to eq(nil)
+ end
+
+ it "returns nothing when the plugin splits no routes" do
+ allow(described_class).to receive(:read_manifest).and_return({ "main" => {} })
+ expect(bundle_for("chat")).to eq(nil)
+ end
+ end
+end