A platform for community discussion. Free, open, simple.
DEV: Introduce bundle-splitting for plugins (#41698)
David Taylor29487ba0bf22
diff --git a/app/views/layouts/_plugin_js.html.erb b/app/views/layouts/_plugin_js.html.erbindex 301d72cc90e..2d3c01bc415 100644--- a/app/views/layouts/_plugin_js.html.erb+++ b/app/views/layouts/_plugin_js.html.erb@@ -18,10 +18,10 @@<% end %><%- end %>- <%- plugin_assets.flat_map { it[:imports] }.compact.uniq.each do |import| %>+ <%- plugin_assets.flat_map { [*it[:imports], it[:route_bundle]] }.compact.uniq.each do |preload| %><linkrel="modulepreload"- href="<%= script_asset_path(import) %>"+ href="<%= script_asset_path(preload) %>"nonce="<%= csp_nonce_placeholder %>"><%- end %>diff --git a/docs/developer-guides/docs/03-code-internals/31-bundle-splitting-for-plugins.md b/docs/developer-guides/docs/03-code-internals/31-bundle-splitting-for-plugins.mdnew file mode 100644index 00000000000..f32b858b62b--- /dev/null+++ b/docs/developer-guides/docs/03-code-internals/31-bundle-splitting-for-plugins.md@@ -0,0 +1,146 @@+---+title: Bundle splitting for plugins+short_title: Bundle splitting+id: bundle-splitting+---++By default, every javascript module in a plugin is loaded when Discourse boots. This guide describes the `staticModules` system, which allows code to be automatically loaded on-demand, thereby improving initial boot times.++This system is currently available to plugins. Theme support will come later++## Enable static modules++Add a `frontend` section to your plugin's `about.json`:++```json+{+ "frontend": {+ "staticModules": true+ }+}+```++Two things happen:++Your components, helpers, modifiers and lib code stop being loaded automatically. Instead, they are only loaded when referenced by an import in other code.++Your routes, controllers & templates move into a separate bundle which loads on demand.++## Group your routes++With `staticModules` on, all the routes in your plugin will go into one bundle. That is usually what you want.++To split further, name a bundle on a route. The route and everything under it go in that bundle:++```js+export default function () {+ this.route("chat", { bundleName: "chat" }, function () {+ this.route("channel", { path: "/c/:title/:id" });+ });++ this.route("chat-settings", { bundleName: "chat-settings" });+}+```++Here `chat` and `chat.channel` load together, and `chat-settings` loads separately.++If a map mounts on another route and all of its routes belong in the same bundle, name it once on the map instead:++```js+export default {+ resource: "admin.adminPlugins.show",+ bundleName: "workflows-admin",++ map() {+ this.route("workflows", { path: "workflows" });+ this.route("workflows-variables", { path: "variables" });+ },+};+```++## Manually loading code on-demand++Bundling is automatic for routes only. Anything imported from an initializer or service is loaded eagerly.++When something is large and rarely used (e.g. a modal that opens on a click), you should import it dynamically instead.++```js+export default async function showNewMessageModal(modal) {+ const { default: ChatModalNewMessage } =+ await import("../components/chat/modal/new-message");++ modal.show(ChatModalNewMessage);+}+```++The path must be a literal, so that the build can find it. To pull in a group of modules with one import, re-export them from a single module and import that.++When lazy-loading components, it normally makes sense to use the `DAsyncContent` wrapper from `ui-kit`. For example:++```gjs+export default class LazyLoader extends Component {+ get complexComponent() {+ return import("./complex-component");+ }++ <template>+ <DAsyncContent @asyncData={{this.complexComponent}}>+ <:content as |componentModule|>+ <componentModule.default />+ </:content>+ </DAsyncContent>+ </template>+}+```++`DAsyncContent` registers its own test waiter. If your plugin uses qunit tests and you call `import()` yourself, wrap the promise in `waitForPromise` so that tests wait for it to load:++```js+import { waitForPromise } from "@ember/test-waiters";++await waitForPromise(import("./my-module"));+```++## Sharing code with other themes and plugins++Enabling `staticModules` puts a boundary around your plugin. If another plugin or theme needs to import one of your modules, list it under `exports` in `about.json`:++```json+{+ "frontend": {+ "staticModules": true,+ "exports": {+ "discourse/components/chat-channel": "discourse/components/chat-channel",+ "discourse/models/chat-channel": "discourse/models/chat-channel"+ }+ }+}+```++The key is the name other plugins import by, and the value is the module behind it. Keeping the two apart means you can move a module without breaking anyone, and you can export the same module under more than one name. Other plugins reach these as `discourse/plugins/<your-plugin>/<key>`.++Adding a module here will cause it to be eagerly loaded when the application boots, and therefore carries a performance cost. Consider providing an asynchronous-friendly entrypoint for any shared components/libs, so that most code is only loaded on the first use.++## Route map syntax++Route maps are statically analyzed during the build, so it's important for them to use simple syntax. Route maps must only include `this.route` calls, with literal string names and options.++This means no conditionals, no loops, and no values read from settings, services, or imports. If the build cannot parse your map, your plugin will fail to compile.++## Compatibility++When using staticModules, you should avoid using these legacy features:++- `templateName`/`controllerName` overrides in routes++- Injecting controllers or routes via `@controller` or `.lookup()`++- Loading components via `.lookup()`++- `requirejs.entries`, `require()`, `define()`++- Loading your plugin's modules from other plugins (except those listed in `exports`)++## Verification++Generated bundles are listed in `app/assets/generated/<your-plugin>/manifest.json` after a build. Discourse will preload the right bundle when a user lands directly on your page.diff --git a/frontend/asset-processor/asset-processor-rollup.js b/frontend/asset-processor/asset-processor-rollup.jsindex 699e2abb716..2cb2d087f2e 100644--- a/frontend/asset-processor/asset-processor-rollup.js+++ b/frontend/asset-processor/asset-processor-rollup.js@@ -19,8 +19,12 @@ import discourseExternalLoader from "./rollup-plugins/discourse-external-loader"import discourseFileSearch from "./rollup-plugins/discourse-file-search";import discourseGjs from "./rollup-plugins/discourse-gjs";import discourseHbs from "./rollup-plugins/discourse-hbs";+import discourseRegisterComponents from "./rollup-plugins/discourse-register-components";+import discourseRouteMaps from "./rollup-plugins/discourse-route-maps";import discourseTerser from "./rollup-plugins/discourse-terser";import discourseVirtualLoader from "./rollup-plugins/discourse-virtual-loader";+import { labelFor, routeNamesFor } from "./rollup-virtual-imports";+import { urlTableFor } from "./route-map-parser";import buildEmberTemplateManipulatorPlugin from "./theme-hbs-ast-transforms";import transformActionSyntax from "./transform-action-syntax";import createVirtualFs from "./virtual-fs";@@ -40,6 +44,10 @@ async function performRollup(modules, opts) {const fs = createVirtualFs(modules, basePath);+ // Filled in by `discourse-route-maps` before the entrypoint is generated from it.+ const routeTables = { bundleByRoute: {}, derived: [] };+ opts.routeTables = routeTables;+const cache = opts.pluginName ? caches.get(opts.pluginName) : false;const result = await rollup({@@ -55,6 +63,12 @@ async function performRollup(modules, opts) {console.info(level, message);},plugins: [+ discourseRouteMaps({+ modules,+ label: labelFor(opts),+ tables: routeTables,+ staticModules: !!opts.frontendConfig?.staticModules,+ }),discourseSourceImports({ fs }),discourseFileSearch(),discourseVirtualLoader({@@ -119,6 +133,7 @@ async function performRollup(modules, opts) {}),discourseHbs(),discourseGjs(),+ discourseRegisterComponents({ basePath }),discourseTerser({ opts }),],});@@ -144,6 +159,55 @@ async function performRollup(modules, opts) {),];+ const routeVirtualPrefix = `${basePath}virtual:route:`;++ // Keyed by the qualified `<entrypoint>:<bundle>` from the module id, because two entrypoints+ // can each have a bundle of the same name.+ const bundleNameByFile = {};+ for (const chunk of bundle.output) {+ if (chunk.facadeModuleId?.startsWith(routeVirtualPrefix)) {+ bundleNameByFile[chunk.fileName] = chunk.facadeModuleId.slice(+ routeVirtualPrefix.length+ );+ }+ }++ // Rollup may hoist the entrypoint module into a shared chunk, so find where it landed.+ function routeBundlesForEntry(entryName) {+ const entrypointModuleId = `${basePath}virtual:entrypoint:${entryName}`;++ const owner = bundle.output.find((chunk) =>+ chunk.moduleIds?.includes(entrypointModuleId)+ );++ const fileNameByBundle = {};++ for (const fileName of owner?.dynamicImports ?? []) {+ const qualified = bundleNameByFile[fileName];++ if (qualified?.startsWith(`${entryName}:`)) {+ fileNameByBundle[qualified.slice(entryName.length + 1)] = fileName;+ }+ }++ // A bundle name means a different bundle in each entrypoint.+ const owned = routeNamesFor(+ opts.entrypoints[entryName].modules,+ routeTables.bundleByRoute+ );+ const urls = urlTableFor(+ routeTables.derived.filter((route) => owned.has(route.name))+ );++ // Already ordered most specific first, which is the order Ruby matches them in.+ return urls+ .filter(({ bundleName }) => fileNameByBundle[bundleName])+ .map(({ bundleName, url }) => ({+ url,+ fileName: fileNameByBundle[bundleName],+ }));+ }+const chunks = Object.fromEntries(bundle.output.filter((c) => c.code)@@ -158,6 +222,9 @@ async function performRollup(modules, opts) {imports: chunk.imports.filter((i) =>bundle.output.find((c) => c.fileName === i)),+ routeBundles: chunk.isEntry+ ? routeBundlesForEntry(chunk.name)+ : undefined,externalPluginImports,},];diff --git a/frontend/asset-processor/rollup-plugins/discourse-register-components.js b/frontend/asset-processor/rollup-plugins/discourse-register-components.jsnew file mode 100644index 00000000000..d90a1070209--- /dev/null+++ b/frontend/asset-processor/rollup-plugins/discourse-register-components.js@@ -0,0 +1,89 @@+import MagicString from "magic-string";+import { stripExtension } from "../rollup-virtual-imports";++const STORE = "discourse/lib/deferred-class-modifications";+const BINDING = "__discourseComponentClass";++// What the resolver's suffix lookup accepts: `components/<name>` at any depth, and never a+// module under `templates/`. `lookupModuleBySuffix` also falls back to `<name>/index`.+const COMPONENT_REGEX = /(^|\/)components\/(.+)$/;+const CLASSIC_TEMPLATE_REGEX = /(^|\/)templates\//;++function componentPathFor(id, basePath) {+ if (!id.startsWith(basePath)) {+ return null;+ }++ const path = id.slice(basePath.length);++ if (CLASSIC_TEMPLATE_REGEX.test(path)) {+ return null;+ }++ const match = path.match(COMPONENT_REGEX);++ return match ? stripExtension(match[2]).replace(/\/index$/, "") : null;+}++// Components are reached by import, so nothing can look one up by name. Each module says what+// it is as it evaluates, which is when a `modifyClass` call that arrived first can be applied.+export default function discourseRegisterComponents({ basePath }) {+ return {+ name: "discourse-register-components",++ transform: {+ // After the gjs and babel transforms, so this parses plain JS.+ order: "post",++ handler(code, id) {+ const path = componentPathFor(id, basePath);++ if (!path) {+ return null;+ }++ const exported = this.parse(code).body.find(+ (node) => node.type === "ExportDefaultDeclaration"+ );++ if (!exported) {+ return null;+ }++ const declaration = exported.declaration;+ const magic = new MagicString(code);++ let binding;++ if (declaration.type === "Identifier") {+ binding = declaration.name;+ } else if (declaration.type === "ClassDeclaration" && declaration.id) {+ binding = declaration.id.name;+ } else {+ binding = BINDING;+ const terminated = code[exported.end - 1] === ";";++ magic.overwrite(+ exported.start,+ declaration.start,+ `const ${binding} = `+ );+ magic.appendLeft(+ exported.end,+ `${terminated ? "" : ";"}\nexport default ${binding};`+ );+ }++ magic.append(+ `\nimport { registerModuleForModifyClass } from "${STORE}";\n` ++ `registerModuleForModifyClass(${JSON.stringify(path)}, ${binding});\n`+ );++ return {+ code: magic.toString(),+ map: magic.generateMap({ hires: true }),+ };+ },+ },+ };+}diff --git a/frontend/asset-processor/rollup-plugins/discourse-register-components.test.mjs b/frontend/asset-processor/rollup-plugins/discourse-register-components.test.mjsnew file mode 100644index 00000000000..68ecf6aa537--- /dev/null+++ b/frontend/asset-processor/rollup-plugins/discourse-register-components.test.mjs@@ -0,0 +1,103 @@+import { parseAst } from "rolldown/parseAst";+import { describe, expect, it } from "vitest";+import discourseRegisterComponents from "./discourse-register-components";++const basePath = "discourse/plugins/chat/";++// Stands in for the `this` a transform hook gets.+const context = { parse: parseAst };++function transform(code, file) {+ const { handler } = discourseRegisterComponents({ basePath }).transform;+ return handler.call(context, code, `${basePath}${file}`)?.code ?? null;+}++describe("discourse-register-components", () => {+ it("registers a named default class under its resolver path", () => {+ const out = transform(+ `export default class ChatHeader {}`,+ "discourse/components/chat-header.gjs"+ );++ expect(out).toContain(+ `registerModuleForModifyClass("chat-header", ChatHeader);`+ );+ expect(out).toContain(`export default class ChatHeader {}`);+ });++ it("gives an anonymous default export a binding to register", () => {+ const out = transform(+ `import Component from "@glimmer/component";\nexport default class extends Component {}`,+ "discourse/components/chat-header.gjs"+ );++ expect(out).toContain(`const __discourseComponentClass = class extends`);+ expect(out).toContain(`export default __discourseComponentClass;`);+ expect(out).toContain(+ `registerModuleForModifyClass("chat-header", __discourseComponentClass);`+ );+ });++ it("handles a default export that is just an identifier", () => {+ const out = transform(+ `const Old = 1;\nexport default Old;`,+ "discourse/components/old-title.gjs"+ );++ expect(out).toContain(`registerModuleForModifyClass("old-title", Old);`);+ });++ it("uses the nested path, and drops an index segment", () => {+ expect(+ transform(+ `export default class A {}`,+ "discourse/components/chat/header/icon.gjs"+ )+ ).toContain(`"chat/header/icon"`);++ expect(+ transform(+ `export default class A {}`,+ "discourse/components/channel-icon/index.gjs"+ )+ ).toContain(`"channel-icon"`);+ });++ it("registers a component wherever the plugin nests it", () => {+ // The resolver finds `components/<name>` as a path suffix at any depth.+ expect(+ transform(`export default class A {}`, "components/survey.gjs")+ ).toContain(`"survey"`);+ expect(+ transform(+ `export default class A {}`,+ "discourse/app/components/panel.gjs"+ )+ ).toContain(`"panel"`);+ });++ it("leaves everything that is not a component alone", () => {+ expect(+ transform(`export default class A {}`, "discourse/routes/chat/channel.js")+ ).toBe(null);+ expect(+ transform(`export const a = 1;`, "discourse/components/no-default.js")+ ).toBe(null);+ });++ it("leaves classic component templates alone", () => {+ // The resolver's suffix trie skips anything under `templates/`.+ expect(+ transform(+ `export default class A {}`,+ "discourse/templates/components/legacy.hbs"+ )+ ).toBe(null);+ expect(+ transform(+ `export default class A {}`,+ "discourse/templates/foo/components/bar.hbs"+ )+ ).toBe(null);+ });+});diff --git a/frontend/asset-processor/rollup-plugins/discourse-route-maps.js b/frontend/asset-processor/rollup-plugins/discourse-route-maps.jsnew file mode 100644index 00000000000..fdaaf8d023e--- /dev/null+++ b/frontend/asset-processor/rollup-plugins/discourse-route-maps.js@@ -0,0 +1,81 @@+import {+ buildRouteTree,+ bundleByRouteFor,+ deriveRoutes,+ parseRouteMap,+} from "../route-map-parser";++// Added to the tree by `Plugin::JsManager`. These build routes in loops, so they are read+// leniently and a plugin's `resource` can mount on them.+const CORE_MAP_REGEX = /^__core__\//;++const ROUTE_MAP_REGEX = /route-map\.js$/;++function isCoreMap(filename) {+ return CORE_MAP_REGEX.test(filename);+}++// Fills `tables` before any module loads, because the entrypoint's source is generated from it.+export default function discourseRouteMaps({+ modules,+ label,+ tables,+ staticModules,+}) {+ return {+ name: "discourse-route-maps",+ buildStart() {+ if (!staticModules) {+ return;+ }++ const filenames = Object.keys(modules).filter((filename) =>+ ROUTE_MAP_REGEX.test(filename)+ );++ // Themes get no core maps, so they cannot resolve a `resource` mount and derive no bundles.+ if (!filenames.some(isCoreMap)) {+ return;+ }++ // Core first, so a plugin extending a core route merges into it rather than creating it.+ filenames.sort(+ (a, b) =>+ Number(!isCoreMap(a)) - Number(!isCoreMap(b)) || a.localeCompare(b)+ );++ const maps = filenames.map((filename) => {+ const source = modules[filename];+ const core = isCoreMap(filename);++ return {+ filename,+ core,+ ...parseRouteMap(this.parse(source), {+ filename,+ source,+ label,+ lenient: core,+ }),+ };+ });++ const { root, unmounted } = buildRouteTree(maps);++ const [unmountable] = unmounted;++ if (unmountable) {+ throw new Error(+ `[${label}] ${unmountable.filename} mounts on "${unmountable.resource}", which is not a route.`+ );+ }++ const derived = deriveRoutes(root);++ Object.assign(tables, {+ derived,+ bundleByRoute: bundleByRouteFor(derived),+ });+ },+ };+}diff --git a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.jsindex 768e19b110d..d2bbd79d2b4 100644--- a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js+++ b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.js@@ -31,6 +31,7 @@ export default function discourseVirtualLoader({? rollupVirtualImports: {"virtual:entrypoint": rollupVirtualImports["virtual:entrypoint"],+ "virtual:route": rollupVirtualImports["virtual:route"],};return {@@ -47,7 +48,8 @@ export default function discourseVirtualLoader({if (availableVirtualImports[source] ||- source.startsWith("virtual:entrypoint:")+ source.startsWith("virtual:entrypoint:") ||+ source.startsWith("virtual:route:")) {return `${basePath}${source}`;}@@ -70,11 +72,32 @@ export default function discourseVirtualLoader({return availableVirtualImports["virtual:entrypoint"](entrypointConfig.modules,opts,- {- basePath,- context: this,- }+ entrypointName);+ } else if (fromBase.startsWith("virtual:route:")) {+ // Entrypoints are built from separate module lists but share one bundle-name+ // namespace, so the entrypoint is part of the id.+ const qualified = fromBase.replace("virtual:route:", "");+ const separator = qualified.indexOf(":");+ const entrypointName = qualified.slice(0, separator);+ const bundleName = qualified.slice(separator + 1);+ const entrypointConfig = entrypoints[entrypointName];++ const source =+ entrypointConfig &&+ availableVirtualImports["virtual:route"](+ entrypointConfig.modules,+ opts,+ bundleName+ );++ if (!source) {+ throw new Error(+ `No route bundle for "${bundleName}" in the ${entrypointName} entrypoint — no route files matched it.`+ );+ }++ return source;} else if (availableVirtualImports[fromBase]) {return availableVirtualImports[fromBase](opts);}diff --git a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjsindex 05127c4642c..3008918cffa 100644--- a/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs+++ b/frontend/asset-processor/rollup-plugins/discourse-virtual-loader.test.mjs@@ -6,11 +6,11 @@ const basePath = "discourse/plugins/chat/";const PLUGIN_API_WRAPPER = `\0${basePath}virtual:attributed-plugin-api`;const API_INITIALIZER_WRAPPER = `\0${basePath}virtual:attributed-api-initializer`;-function loader(isTheme = false) {+function loader(isTheme = false, entrypoints = {}, opts = {}) {return discourseVirtualLoader({basePath,- entrypoints: {},- opts: { pluginName: "chat" },+ entrypoints,+ opts: { pluginName: "chat", ...opts },isTheme,});}@@ -54,3 +54,44 @@ test("plugin builds can load the wrappers but not virtual:theme", () => {expect(load(PLUGIN_API_WRAPPER)).toContain("_INTERNAL_SOURCE_KEY");expect(load(`${basePath}virtual:theme`)).toBeUndefined();});++test("loads a route bundle from its own entrypoint, not another with the same name", () => {+ const { load } = loader(+ false,+ {+ main: { modules: ["discourse/routes/chat.js"] },+ admin: { modules: ["discourse/routes/admin-plugins/show/hooks.js"] },+ },+ {+ frontendConfig: { staticModules: true },+ routeTables: {+ bundleByRoute: {+ chat: "default",+ "admin-plugins.show.hooks": "default",+ },+ },+ }+ );++ expect(load(`${basePath}virtual:route:main:default`)).toContain(+ '"discourse/routes/chat":'+ );+ expect(load(`${basePath}virtual:route:admin:default`)).toContain(+ '"discourse/routes/admin-plugins/show/hooks":'+ );+});++test("a route bundle missing from the named entrypoint is an error", () => {+ const { load } = loader(+ false,+ { main: { modules: ["discourse/routes/chat.js"] } },+ {+ frontendConfig: { staticModules: true },+ routeTables: { bundleByRoute: { chat: "default" } },+ }+ );++ expect(() => load(`${basePath}virtual:route:admin:default`)).toThrow(+ /No route bundle for "default" in the admin entrypoint/+ );+});diff --git a/frontend/asset-processor/rollup-virtual-imports.js b/frontend/asset-processor/rollup-virtual-imports.jsindex f185575235a..81360c55cd3 100644--- a/frontend/asset-processor/rollup-virtual-imports.js+++ b/frontend/asset-processor/rollup-virtual-imports.js@@ -9,79 +9,299 @@ const SUPPORTED_FILE_EXTENSIONS = [const IS_CONNECTOR_REGEX = /(^|\/)connectors\//;-export default {- "virtual:entrypoint": (moduleFilenames, { themeId, pluginName }) => {- const label = pluginName ? `PLUGIN ${pluginName}` : `THEME ${themeId}`;- const imports = [];- const entries = [];- const warnings = [];-- const exportedModules = new Set();-- let i = 1;- for (const moduleFilename of moduleFilenames) {- // Type-only declaration files have no runtime module to export.- if (moduleFilename.endsWith(".d.ts")) {- continue;- }+// Scanned by name at runtime, at any depth, so these have to stay registered with `define()`.+const EAGER_DIRECTORIES = [+ "connectors",+ "services",+ "models",+ "adapters",+ "discourse-markdown",+ "markdown-it",+ "pre-initializers",+ "initializers",+ "api-initializers",+ "instance-initializers",+];++const EAGER_DIRECTORY_REGEX = new RegExp(+ `(^|/)(${EAGER_DIRECTORIES.join("|")})/`+);++function isEagerModule(compatModuleName) {+ return (+ EAGER_DIRECTORY_REGEX.test(compatModuleName) ||+ // `mapRoutes` matches on the suffix alone.+ /route-map$/.test(compatModuleName)+ );+}++export function stripExtension(filename) {+ return filename.replace(/\.[^\.]+(\.es6)?$/, "");+}++function normalizeModules(moduleFilenames, label) {+ const records = [];+ const warnings = [];+ const seen = new Set();++ for (const moduleFilename of moduleFilenames) {+ if (moduleFilename.endsWith(".d.ts")) {+ continue;+ }- if (- !SUPPORTED_FILE_EXTENSIONS.some((ext) => moduleFilename.endsWith(ext))- ) {- // Unsupported file type. Log a warning and skip- warnings.push(- `console.warn("[${label}] Unsupported file type: ${moduleFilename}");`+ if (+ !SUPPORTED_FILE_EXTENSIONS.some((ext) => moduleFilename.endsWith(ext))+ ) {+ warnings.push(+ `console.warn("[${label}] Unsupported file type: ${moduleFilename}");`+ );+ continue;+ }++ const filenameWithoutExtension = stripExtension(moduleFilename);++ let compatModuleName = filenameWithoutExtension;++ if (moduleFilename.match(IS_CONNECTOR_REGEX)) {+ const isTemplate = moduleFilename.endsWith(".hbs");+ const isInTemplatesDirectory = moduleFilename.match(/(^|\/)templates\//);++ if (isTemplate && !isInTemplatesDirectory) {+ compatModuleName = compatModuleName.replace(+ IS_CONNECTOR_REGEX,+ "$1templates/connectors/");- continue;+ } else if (!isTemplate && isInTemplatesDirectory) {+ compatModuleName = compatModuleName.replace(/(^|\/)templates\//, "$1");}+ }- const filenameWithoutExtension = moduleFilename.replace(- /\.[^\.]+(\.es6)?$/,- ""- );+ const importPath = filenameWithoutExtension.match(IS_CONNECTOR_REGEX)+ ? moduleFilename+ : filenameWithoutExtension;++ if (seen.has(importPath)) {+ continue;+ }+ seen.add(importPath);++ records.push({ importPath, compatModuleName });+ }++ return { records, warnings };+}++// Anchored to the top-level segment: a component under `components/chat/routes/` is not a route.+const ROUTE_FILE_REGEX = /^[^/]+\/(routes|controllers|templates)\/(.+)$/;++function routeNameFor(compatModuleName) {+ const match = compatModuleName.match(ROUTE_FILE_REGEX);++ if (!match) {+ return null;+ }++ const [, type, path] = match;++ // Discourse nests connectors and classic component templates under `templates/` too.+ if (+ type === "templates" &&+ (path.startsWith("connectors/") || path.startsWith("components/"))+ ) {+ return null;+ }++ return path.split("/").join(".");+}++// Ember creates these without a `this.route` call, so no route map names them.+const IMPLICIT_ROUTE_SUFFIXES = ["index", "loading", "error"];++function bundleNameFor(routeName, bundleByRoute) {+ let name = routeName;++ while (name) {+ if (bundleByRoute[name]) {+ return bundleByRoute[name];+ }++ const dot = name.lastIndexOf(".");++ if (!IMPLICIT_ROUTE_SUFFIXES.includes(name.slice(dot + 1))) {+ return null;+ }++ name = dot === -1 ? "" : name.slice(0, dot);+ }++ return null;+}- let compatModuleName = filenameWithoutExtension;-- if (moduleFilename.match(IS_CONNECTOR_REGEX)) {- const isTemplate = moduleFilename.endsWith(".hbs");- const isInTemplatesDirectory =- moduleFilename.match(/(^|\/)templates\//);-- if (isTemplate && !isInTemplatesDirectory) {- compatModuleName = compatModuleName.replace(- IS_CONNECTOR_REGEX,- "$1templates/connectors/"- );- } else if (!isTemplate && isInTemplatesDirectory) {- compatModuleName = compatModuleName.replace(- /(^|\/)templates\//,- "$1"- );+export function routeBundlesFor(records, bundleByRoute) {+ if (!bundleByRoute) {+ return [];+ }++ const bundles = new Map();++ for (const record of records) {+ const routeName = routeNameFor(record.compatModuleName);+ const bundleName = routeName && bundleNameFor(routeName, bundleByRoute);++ if (!bundleName) {+ continue;+ }++ let bundle = bundles.get(bundleName);++ if (!bundle) {+ bundle = { bundleName, names: new Set(), records: [] };+ bundles.set(bundleName, bundle);+ }++ bundle.names.add(routeName);+ bundle.records.push(record);+ }++ return [...bundles.values()].map((bundle) => ({+ ...bundle,+ names: [...bundle.names].sort(),+ }));+}++export function routeNamesFor(moduleFilenames, bundleByRoute) {+ const { records } = normalizeModules(moduleFilenames);++ return new Set(+ routeBundlesFor(records, bundleByRoute).flatMap((bundle) => bundle.names)+ );+}++export function labelFor({ pluginName, themeId }) {+ return pluginName ? `PLUGIN ${pluginName}` : `THEME ${themeId}`;+}++function renderImports(records) {+ const identifiers = new Map(+ records.map((record, i) => [record, `Mod${i + 1}`])+ );++ const lines = records.map(+ (record) =>+ `import * as ${identifiers.get(record)} from "./${record.importPath}";`+ );++ return { lines, identifiers };+}++function renderMap(name, records, identifiers) {+ return [+ `const ${name} = {`,+ ...records.map(+ (record) => ` "${record.compatModuleName}": ${identifiers.get(record)},`+ ),+ "};",+ ];+}++export default {+ "virtual:entrypoint": (moduleFilenames, opts, entrypointName) => {+ const { frontendConfig } = opts;++ const { records, warnings } = normalizeModules(+ moduleFilenames,+ labelFor(opts)+ );++ // QUnit only finds a test by running the module that registers it, so tests stay eager.+ if (entrypointName === "test" || !frontendConfig?.staticModules) {+ const { lines, identifiers } = renderImports(records);++ return [+ ...lines,+ ...warnings,+ ...renderMap("compatModules", records, identifiers),+ "export { compatModules };",+ "export default compatModules;",+ "",+ ].join("\n");+ }++ // `exports` maps the name other bundles import by to the module behind it. That module can be+ // named with or without the `/index` a colocated component adds.+ const publicNamesByPath = new Map();++ for (const [publicName, internalName] of Object.entries(+ frontendConfig.exports ?? {}+ )) {+ const path = stripExtension(internalName);++ for (const candidate of [path, `${path}/index`]) {+ if (!publicNamesByPath.has(candidate)) {+ publicNamesByPath.set(candidate, []);}+ publicNamesByPath.get(candidate).push(publicName);}+ }- const importPath = filenameWithoutExtension.match(IS_CONNECTOR_REGEX)- ? moduleFilename- : filenameWithoutExtension;+ const bundles = routeBundlesFor(records, opts.routeTables?.bundleByRoute);- if (exportedModules.has(importPath)) {- continue;- }- exportedModules.add(importPath);+ const eager = records.filter((record) =>+ isEagerModule(record.compatModuleName)+ );- imports.push(`import * as Mod${i} from "./${importPath}";`);- entries.push(` "${compatModuleName}": Mod${i},`);+ // One module can be exported under several names.+ const exported = records.flatMap((record) =>+ (publicNamesByPath.get(stripExtension(record.importPath)) ?? []).map(+ (publicName) => ({ publicName, record })+ )+ );- i += 1;- }+ const imported = [+ ...new Set([...eager, ...exported.map(({ record }) => record)]),+ ];+ const { lines, identifiers } = renderImports(imported);return [- ...imports,+ ...lines,...warnings,- "const compatModules = {",- ...entries,+ ...renderMap("compatModules", eager, identifiers),+ "const pluginExports = {",+ ...exported.map(+ ({ publicName, record }) =>+ ` "${publicName}": ${identifiers.get(record)},`+ ),"};",- "export default compatModules;",+ "export const routes = [",+ ...bundles.map(+ (bundle) =>+ ` { names: ${JSON.stringify(bundle.names)},` ++ ` load: () => import("virtual:route:${entrypointName}:${bundle.bundleName}") },`+ ),+ "];",+ "export { compatModules };",+ "export default pluginExports;",+ "",+ ].join("\n");+ },+ // The default export goes to `Resolver#addModules`, so it must be a plain module map.+ "virtual:route": (moduleFilenames, opts, bundleName) => {+ const { records } = normalizeModules(moduleFilenames, labelFor(opts));+ const bundle = routeBundlesFor(+ records,+ opts.routeTables?.bundleByRoute+ ).find((candidate) => candidate.bundleName === bundleName);++ if (!bundle) {+ return null;+ }++ const { lines, identifiers } = renderImports(bundle.records);++ return [+ ...lines,+ ...renderMap("routeCompatModules", bundle.records, identifiers),+ "export default routeCompatModules;","",].join("\n");},diff --git a/frontend/asset-processor/rollup-virtual-imports.test.mjs b/frontend/asset-processor/rollup-virtual-imports.test.mjsindex df6f90175c0..d729813207f 100644--- a/frontend/asset-processor/rollup-virtual-imports.test.mjs+++ b/frontend/asset-processor/rollup-virtual-imports.test.mjs@@ -1,44 +1,411 @@-/* eslint-disable qunit/require-expect */-import { expect, test } from "vitest";-import { privateVirtualImports } from "./rollup-virtual-imports.js";+import { describe, expect, it } from "vitest";+import rollupVirtualImports, {+ privateVirtualImports,+ routeNamesFor,+} from "./rollup-virtual-imports";-test("attributed plugin-api module wraps withPluginApi for a plugin", () => {- const code = privateVirtualImports["virtual:attributed-plugin-api"]({- pluginName: "chat",+function entrypoint(moduleFilenames, opts = {}, entrypointName = "main") {+ return rollupVirtualImports["virtual:entrypoint"](+ moduleFilenames,+ {+ pluginName: "chat",+ ...opts,+ },+ entrypointName+ );+}++const MODULES = [+ "discourse/components/chat-channel.gjs",+ "discourse/services/chat.js",+ "discourse/models/channel.js",+ "discourse/adapters/chat.js",+ "discourse/initializers/chat-setup.js",+ "discourse/api-initializers/chat.js",+ "discourse/pre-initializers/chat-early.js",+ "discourse/chat-route-map.js",+ "discourse/connectors/user-menu/chat.gjs",+ "discourse/routes/chat/channel.js",+ "discourse/controllers/chat/channel.js",+ "discourse/templates/chat/channel.hbs",+ "discourse/lib/chat-utils.js",+ "discourse/helpers/format-chat-date.js",+];++describe("virtual:entrypoint", () => {+ describe("without staticModules", () => {+ it("eagerly imports every module and exports one object under both names", () => {+ const output = entrypoint(MODULES);++ for (const filename of MODULES) {+ // Connectors keep their extension; everything else is stripped.+ const importPath = filename.includes("/connectors/")+ ? filename+ : filename.replace(/\.\w+$/, "");+ expect(output, filename).toContain(`from "./${importPath}"`);+ }++ expect(output).toContain("export { compatModules };");+ expect(output).toContain("export default compatModules;");+ expect(output).not.toContain("pluginExports");+ });++ it("skips type declarations and warns about unsupported files", () => {+ const output = entrypoint([+ "discourse/lib/types.d.ts",+ "discourse/lib/notes.md",+ "discourse/services/chat.js",+ ]);++ expect(output).not.toContain("types.d.ts");+ expect(output).toContain("Unsupported file type: discourse/lib/notes.md");+ expect(output).toContain('from "./discourse/services/chat"');+ });});- expect(code).toContain(- 'import { _INTERNAL_SOURCE_KEY } from "discourse/lib/api";'- );- expect(code).toContain(- 'import { withPluginApi as _withPluginApi } from "discourse/lib/plugin-api";'- );- expect(code).toContain('Object.freeze({"type":"plugin","name":"chat"})');- expect(code).toContain("export function withPluginApi(...args)");- expect(code).toContain("[_INTERNAL_SOURCE_KEY]: SOURCE");- expect(code).toContain("return _withPluginApi(...args);");-});+ describe("with staticModules", () => {+ const output = entrypoint(MODULES, {+ frontendConfig: {+ staticModules: true,+ exports: {+ "discourse/components/chat-channel":+ "discourse/components/chat-channel.gjs",+ "discourse/models/channel": "discourse/models/channel.js",+ },+ },+ });++ const compatModules = output.slice(+ output.indexOf("const compatModules"),+ output.indexOf("const pluginExports")+ );+ const pluginExports = output.slice(output.indexOf("const pluginExports"));++ it("registers everything Discourse resolves by name", () => {+ for (const name of [+ "discourse/services/chat",+ "discourse/models/channel",+ "discourse/adapters/chat",+ "discourse/initializers/chat-setup",+ "discourse/api-initializers/chat",+ "discourse/pre-initializers/chat-early",+ "discourse/chat-route-map",+ "discourse/connectors/user-menu/chat",+ ]) {+ expect(compatModules, name).toContain(`"${name}":`);+ }+ });++ it("registers them wherever the plugin nests them", () => {+ // The resolver matches a path suffix, and the connector and markdown scans match anywhere,+ // so a plugin is free to keep these below the top level.+ const nested = entrypoint(+ [+ "discourse/app/services/voice-webrtc.js",+ "discourse/lib/discourse-markdown/spoiler.js",+ "discourse-markdown/graphviz.js",+ "pre-initializers/h-captcha.js",+ "discourse/templates/connectors/user-menu/chat.hbs",+ ],+ { frontendConfig: { staticModules: true } }+ );++ for (const name of [+ "discourse/app/services/voice-webrtc",+ "discourse/lib/discourse-markdown/spoiler",+ "discourse-markdown/graphviz",+ "pre-initializers/h-captcha",+ "discourse/templates/connectors/user-menu/chat",+ ]) {+ expect(nested, name).toContain(`"${name}":`);+ }+ });++ it("does not register routes, controllers or route templates", () => {+ for (const name of [+ "discourse/routes/chat/channel",+ "discourse/controllers/chat/channel",+ "discourse/templates/chat/channel",+ ]) {+ expect(compatModules, name).not.toContain(`"${name}":`);+ }+ });-test("attributed api-initializer module wraps apiInitializer for a theme", () => {- const code = privateVirtualImports["virtual:attributed-api-initializer"]({- themeId: 42,+ it("leaves invokables and lib code to be statically imported", () => {+ // Registering these would stop them tree-shaking.+ expect(compatModules).not.toContain(+ '"discourse/helpers/format-chat-date"'+ );+ expect(compatModules).not.toContain('"discourse/lib/chat-utils"');+ expect(compatModules).not.toContain(+ '"discourse/components/chat-channel"'+ );+ });++ it("exports only the declared modules as the cross-bundle API", () => {+ expect(pluginExports).toContain('"discourse/components/chat-channel":');+ expect(pluginExports).toContain('"discourse/models/channel":');+ expect(pluginExports).not.toContain('"discourse/services/chat":');+ expect(output).toContain("export default pluginExports;");+ });++ it("keys the export by its public name, not the module behind it", () => {+ const aliased = entrypoint(MODULES, {+ frontendConfig: {+ staticModules: true,+ exports: {+ "discourse/chat-channel": "discourse/components/chat-channel.gjs",+ "discourse/channel": "discourse/components/chat-channel.gjs",+ },+ },+ });+ const map = aliased.slice(aliased.indexOf("const pluginExports"));++ expect(map).toContain('"discourse/chat-channel":');+ expect(map).toContain('"discourse/channel":');+ expect(map).not.toContain('"discourse/components/chat-channel":');+ });++ it("imports a module shared and registered only once", () => {+ // `discourse/models/channel` is both eager and shared.+ const imports = output+ .split("\n")+ .filter((line) => line.includes('from "./discourse/models/channel"'));+ expect(imports).toHaveLength(1);+ });++ it("does not import modules which are neither eager nor shared", () => {+ expect(output).not.toContain('from "./discourse/lib/chat-utils"');+ expect(output).not.toContain(+ 'from "./discourse/helpers/format-chat-date"'+ );+ });++ it("emits no routes export when nothing is split", () => {+ expect(output).toContain("export const routes = [\n];");+ });});- expect(code).toContain(- 'import { apiInitializer as _apiInitializer } from "discourse/lib/api";'- );- expect(code).toContain('Object.freeze({"type":"theme","id":42})');- expect(code).toContain("export function apiInitializer(...args)");- expect(code).toContain("return _apiInitializer(...args);");+ describe("test entrypoint", () => {+ const TEST_MODULES = [+ "acceptance/chat-message-test.gjs",+ "components/chat-channel-test.gjs",+ "unit/lib/chat-utils-test.js",+ "unit/services/chat-test.js",+ "integration/components/user-menu/chat-test.gjs",+ "helpers/chat-fixtures.js",+ ];++ it("eagerly imports every module despite staticModules", () => {+ const output = entrypoint(+ TEST_MODULES,+ { frontendConfig: { staticModules: true, exports: {} } },+ "test"+ );++ for (const filename of TEST_MODULES) {+ const importPath = filename.replace(/\.\w+$/, "");+ expect(output, filename).toContain(`from "./${importPath}"`);+ }++ expect(output).toContain("export default compatModules;");+ expect(output).not.toContain("pluginExports");+ });++ it("still tree-shakes the same modules for a non-test entrypoint", () => {+ const output = entrypoint(+ TEST_MODULES,+ { frontendConfig: { staticModules: true, exports: {} } },+ "main"+ );++ expect(output).not.toContain('from "./components/chat-channel-test"');+ expect(output).not.toContain('from "./unit/lib/chat-utils-test"');+ });+ });++ describe("route bundles", () => {+ const ROUTE_MODULES = [+ "discourse/routes/chat.js",+ "discourse/routes/chat/channel.js",+ "discourse/controllers/chat/channel.js",+ "discourse/templates/chat/channel.hbs",+ "discourse/routes/chat/visualizer.js",+ "discourse/templates/chat/visualizer.hbs",+ "discourse/routes/browse.js",+ "discourse/services/chat.js",+ "discourse/templates/connectors/user-menu/chat.hbs",+ "discourse/templates/components/chat-message.hbs",+ // Chat really has these: components in a directory called `routes`.+ "discourse/components/chat/routes/channel.gjs",+ ];++ const frontendConfig = { staticModules: true };++ // `chat.visualizer` names its own bundle, so it is not swept into `chat`.+ const routeTables = {+ bundleByRoute: {+ chat: "chat",+ "chat.channel": "chat",+ "chat.visualizer": "visualizer",+ },+ };++ const output = entrypoint(ROUTE_MODULES, { frontendConfig, routeTables });++ it("groups routes into a bundle per declared name", () => {+ expect(output).toContain(+ `{ names: ["chat.visualizer"], load: () => import("virtual:route:main:visualizer") },`+ );+ expect(output).toContain(+ `{ names: ["chat","chat.channel"], load: () => import("virtual:route:main:chat") },`+ );+ });++ it("puts an implicit index route in its parent's bundle", () => {+ const withIndex = entrypoint(+ [...ROUTE_MODULES, "discourse/routes/chat/index.js"],+ { frontendConfig, routeTables }+ );++ expect(withIndex).toContain(+ `{ names: ["chat","chat.channel","chat.index"], load: () => import("virtual:route:main:chat") },`+ );+ });++ it("keeps a bundle name in one entrypoint clear of the same name in another", () => {+ // Both entrypoints put their unnamed routes in `default`, and each is built from its own+ // module list.+ const shared = { bundleByRoute: { chat: "default", admin: "default" } };++ const main = entrypoint(+ ["discourse/routes/chat.js"],+ { frontendConfig, routeTables: shared },+ "main"+ );+ const admin = entrypoint(+ ["discourse/routes/admin.js"],+ { frontendConfig, routeTables: shared },+ "admin"+ );++ expect(main).toContain(+ `{ names: ["chat"], load: () => import("virtual:route:main:default") },`+ );+ expect(admin).toContain(+ `{ names: ["admin"], load: () => import("virtual:route:admin:default") },`+ );+ });++ it("keeps split route files out of the eager set", () => {+ const compatModules = output.slice(+ output.indexOf("const compatModules"),+ output.indexOf("const pluginExports")+ );++ expect(compatModules).not.toContain('"discourse/routes/chat"');+ expect(compatModules).not.toContain('"discourse/routes/chat/channel"');+ expect(compatModules).not.toContain('"discourse/templates/chat/channel"');++ // A route no map names is not registered either.+ expect(compatModules).not.toContain('"discourse/routes/browse":');++ expect(compatModules).toContain('"discourse/services/chat":');+ });++ it("does not mistake connectors or component templates for routes", () => {+ expect(output).not.toContain('import("virtual:route:connectors');+ expect(output).not.toContain('import("virtual:route:components');+ expect(output).not.toContain('"connectors.user-menu.chat"');+ expect(output).not.toContain('"components.chat-message"');+ });++ it("only treats top-level routes/controllers/templates as routes", () => {+ const compatModules = output.slice(+ output.indexOf("const compatModules"),+ output.indexOf("const pluginExports")+ );++ expect(compatModules).not.toContain(+ '"discourse/components/chat/routes/channel"'+ );+ expect(output).not.toContain('import("virtual:route:channel")');+ });++ it("renders a route bundle as a plain module map", () => {+ const bundle = rollupVirtualImports["virtual:route"](+ ROUTE_MODULES,+ { pluginName: "chat", frontendConfig, routeTables },+ "chat"+ );++ expect(bundle).toContain('"discourse/routes/chat":');+ expect(bundle).toContain('"discourse/routes/chat/channel":');+ expect(bundle).toContain('"discourse/controllers/chat/channel":');+ expect(bundle).toContain('"discourse/templates/chat/channel":');+ expect(bundle).toContain("export default routeCompatModules;");++ // The separately-split child does not belong to the parent bundle.+ expect(bundle).not.toContain('"discourse/routes/chat/visualizer":');+ });+ });});-test("the source lands on opts, after the legacy version argument", () => {- const code = privateVirtualImports["virtual:attributed-plugin-api"]({- pluginName: "chat",+describe("privateVirtualImports", () => {+ it("wraps withPluginApi for a plugin", () => {+ const code = privateVirtualImports["virtual:attributed-plugin-api"]({+ pluginName: "chat",+ });++ expect(code).toContain(+ 'import { _INTERNAL_SOURCE_KEY } from "discourse/lib/api";'+ );+ expect(code).toContain(+ 'import { withPluginApi as _withPluginApi } from "discourse/lib/plugin-api";'+ );+ expect(code).toContain('Object.freeze({"type":"plugin","name":"chat"})');+ expect(code).toContain("export function withPluginApi(...args)");+ expect(code).toContain("[_INTERNAL_SOURCE_KEY]: SOURCE");+ expect(code).toContain("return _withPluginApi(...args);");});- expect(code).toContain('typeof args[0] === "string" ? 2 : 1');- expect(code).toContain(- "args[optsIndex] = { ...args[optsIndex], [_INTERNAL_SOURCE_KEY]: SOURCE };"- );+ it("wraps apiInitializer for a theme", () => {+ const code = privateVirtualImports["virtual:attributed-api-initializer"]({+ themeId: 42,+ });++ expect(code).toContain(+ 'import { apiInitializer as _apiInitializer } from "discourse/lib/api";'+ );+ expect(code).toContain('Object.freeze({"type":"theme","id":42})');+ expect(code).toContain("export function apiInitializer(...args)");+ expect(code).toContain("return _apiInitializer(...args);");+ });++ it("puts the source on opts, after the legacy version argument", () => {+ const code = privateVirtualImports["virtual:attributed-plugin-api"]({+ pluginName: "chat",+ });++ expect(code).toContain('typeof args[0] === "string" ? 2 : 1');+ expect(code).toContain(+ "args[optsIndex] = { ...args[optsIndex], [_INTERNAL_SOURCE_KEY]: SOURCE };"+ );+ });+});++describe("routeNamesFor", () => {+ it("reports only the routes a module list backs", () => {+ const bundleByRoute = { chat: "default", "admin.hooks": "default" };++ expect([+ ...routeNamesFor(["discourse/routes/chat.js"], bundleByRoute),+ ]).toEqual(["chat"]);+ expect([+ ...routeNamesFor(["discourse/templates/admin/hooks.hbs"], bundleByRoute),+ ]).toEqual(["admin.hooks"]);+ expect([+ ...routeNamesFor(["discourse/services/chat.js"], bundleByRoute),+ ]).toEqual([]);+ });});diff --git a/frontend/asset-processor/route-map-parser.js b/frontend/asset-processor/route-map-parser.jsnew file mode 100644index 00000000000..8797c3bd325--- /dev/null+++ b/frontend/asset-processor/route-map-parser.js@@ -0,0 +1,358 @@+// Route maps are parsed, never run: plugin and theme code is not trusted enough to evaluate here.+// `mapping-router.js` is the reference for what a map means.++// `BareRouter#lazyRoute` dasherizes before matching, so derived names have to agree.+function dasherize(value) {+ return value+ .replace(/([a-z\d])([A-Z])/g, "$1-$2")+ .replace(/_/g, "-")+ .toLowerCase();+}++function positionOf(source, offset) {+ const upTo = source.slice(0, offset);+ const line = upTo.split("\n").length;+ return { line, column: offset - (upTo.lastIndexOf("\n") + 1) + 1 };+}++function fail(context, node, message) {+ const { line, column } = positionOf(context.source, node.start);+ throw new Error(+ `[${context.label}] ${context.filename}:${line}:${column} — ${message}`+ );+}++const FUNCTION_TYPES = ["FunctionExpression", "FunctionDeclaration"];++function keyName(property) {+ return property.key.name ?? property.key.value;+}++function isThisRouteCall(node) {+ return (+ node.type === "CallExpression" &&+ node.callee.type === "MemberExpression" &&+ !node.callee.computed &&+ node.callee.object.type === "ThisExpression" &&+ node.callee.property.name === "route"+ );+}++function readOptions(context, node) {+ const options = {};++ for (const property of node.properties) {+ if (property.type !== "Property" || property.computed) {+ fail(context, property, "route options must be plain literal properties");+ }++ if (property.value.type !== "Literal") {+ fail(+ context,+ property.value,+ `option "${keyName(property)}" must be a literal`+ );+ }++ options[keyName(property)] = property.value.value;+ }++ return options;+}++function readRouteCall(context, node) {+ const [nameNode, ...rest] = node.arguments;++ if (nameNode?.type !== "Literal" || typeof nameNode.value !== "string") {+ fail(context, nameNode ?? node, "route name must be a string literal");+ }++ let options = {};+ let children = [];++ for (const argument of rest) {+ if (argument.type === "ObjectExpression") {+ options = readOptions(context, argument);+ } else if (FUNCTION_TYPES.includes(argument.type)) {+ children = readBody(context, argument.body);+ } else if (argument.type === "ArrowFunctionExpression") {+ // `this` inside an arrow is the enclosing scope, so the DSL cannot work there.+ fail(context, argument, "route callback must be a function expression");+ } else {+ fail(context, argument, "unexpected argument to this.route");+ }+ }++ return { name: nameNode.value, options, children };+}++function readBody(context, block) {+ const routes = [];++ for (const statement of block.body) {+ if (+ statement.type === "ExpressionStatement" &&+ isThisRouteCall(statement.expression)+ ) {+ routes.push(readRouteCall(context, statement.expression));+ continue;+ }++ // Core's maps build routes in loops. Those routes are missing from the derived tree.+ if (context.lenient) {+ continue;+ }++ fail(+ context,+ statement,+ "a route map may only contain this.route calls with literal arguments"+ );+ }++ return routes;+}++export function parseRouteMap(+ ast,+ { filename, source, label, lenient = false }+) {+ const context = { filename, source, label, lenient };++ const exported = ast.body.find(+ (node) => node.type === "ExportDefaultDeclaration"+ );++ if (!exported) {+ fail(context, ast.body[0] ?? { start: 0 }, "no default export");+ }++ const declaration = exported.declaration;++ if (FUNCTION_TYPES.includes(declaration.type)) {+ return {+ resource: null,+ bundleName: null,+ routes: readBody(context, declaration.body),+ };+ }++ if (declaration.type !== "ObjectExpression") {+ fail(+ context,+ declaration,+ "a route map must export a function or an object"+ );+ }++ const resource = declaration.properties.find(+ (property) =>+ property.type === "Property" && keyName(property) === "resource"+ );+ const map = declaration.properties.find(+ (property) => property.type === "Property" && keyName(property) === "map"+ );+ const bundleName = declaration.properties.find(+ (property) =>+ property.type === "Property" && keyName(property) === "bundleName"+ );++ if (+ bundleName &&+ (bundleName.value.type !== "Literal" ||+ typeof bundleName.value.value !== "string")+ ) {+ fail(context, bundleName.value, "`bundleName` must be a string literal");+ }++ if (!resource || resource.value.type !== "Literal") {+ fail(+ context,+ resource ?? declaration,+ "`resource` must be a string literal"+ );+ }++ if (!map || !FUNCTION_TYPES.includes(map.value.type)) {+ fail(context, map ?? declaration, "`map` must be a function");+ }++ // `mapping-router.js` reads only `resource` and `map`, so the rest is ours to use.+ return {+ resource: resource.value.value,+ bundleName: bundleName?.value.value ?? null,+ routes: readBody(context, map.value.body),+ };+}++// Mirrors `RouteNode`: a child with the same name is merged into, so several maps can extend+// one route.+function addRoutes(parent, routes, core, bundleName = null) {+ for (const route of routes) {+ let node = parent.byName.get(route.name);++ if (!node) {+ const options = { ...route.options };++ if (bundleName && options.bundleName === undefined) {+ options.bundleName = bundleName;+ }++ node = {+ name: route.name,+ options,+ core,+ byName: new Map(),+ };+ parent.byName.set(route.name, node);+ }++ // Only the map's own top level is seeded, because everything below it inherits.+ addRoutes(node, route.children, core);+ }+}++// `resource` walks the names given to `this.route`, which are not route names:+// `admin.adminPlugins.show` reaches a route named `adminPlugins.show`.+function findPath(root, resource) {+ let node = root;++ for (const segment of resource.split(".")) {+ node = node.byName.get(segment);++ if (!node) {+ return null;+ }+ }++ return node;+}++export function buildRouteTree(maps) {+ const root = { name: "root", options: {}, byName: new Map() };+ const unmounted = [];++ for (const map of maps.filter((candidate) => !candidate.resource)) {+ addRoutes(root, map.routes, map.core, map.bundleName);+ }++ for (const map of maps.filter((candidate) => candidate.resource)) {+ const node = findPath(root, map.resource);++ if (node) {+ addRoutes(node, map.routes, map.core, map.bundleName);+ } else {+ unmounted.push(map);+ }+ }++ return { root, unmounted };+}++function joinUrl(parent, path) {+ return [...parent.split("/"), ...path.split("/")].filter(Boolean).join("/");+}++// `Plugin::JsManager.url_glob_pattern` reads `*` as one segment and `**` as one or more.+function globFor(url) {+ return url+ .split("/")+ .map((segment) => {+ if (segment.startsWith("*")) {+ return "**";+ }++ return segment.startsWith(":") ? "*" : segment;+ })+ .join("/");+}++// Nothing a plugin declares is eager, so a route naming no bundle still falls into one.+const DEFAULT_BUNDLE_NAME = "default";++export function deriveRoutes(root) {+ const derived = [];++ function walk(nodes, parent) {+ for (const node of nodes) {+ const { options } = node;++ const name =+ options.resetNamespace || !parent.name+ ? node.name+ : `${parent.name}.${node.name}`;+ const url = joinUrl(parent.url, options.path ?? node.name);+ const bundleName =+ options.bundleName ??+ parent.bundleName ??+ (node.core ? null : DEFAULT_BUNDLE_NAME);++ derived.push({+ name: dasherize(name),+ url,+ bundleName,+ });++ walk(node.byName.values(), { name, url, bundleName });+ }+ }++ walk(root.byName.values(), { name: "", url: "", bundleName: null });++ return derived;+}++function literalSegments(glob) {+ return glob.split("/").filter((segment) => !/^\*\*?$/.test(segment)).length;+}++function segmentCount(glob) {+ return glob.split("/").length;+}++// A glob a shallower one in the same bundle already covers adds nothing.+function urlsFor(globs) {+ const sorted = [...new Set(globs)].sort();++ return sorted.filter(+ (glob) =>+ !sorted.some((other) => other !== glob && glob.startsWith(`${other}/`))+ );+}++export function bundleByRouteFor(derived) {+ const bundleByRoute = {};++ for (const route of derived) {+ if (route.bundleName) {+ bundleByRoute[route.name] = route.bundleName;+ }+ }++ return bundleByRoute;+}++// Urls come out most specific first, so the first match wins whatever order routes were declared.+export function urlTableFor(derived) {+ const globsByBundle = new Map();++ for (const route of derived) {+ if (!route.bundleName) {+ continue;+ }++ const globs = globsByBundle.get(route.bundleName) ?? [];+ globs.push(globFor(route.url));+ globsByBundle.set(route.bundleName, globs);+ }++ return [...globsByBundle]+ .flatMap(([bundleName, globs]) =>+ urlsFor(globs).map((glob) => ({ bundleName, url: `${glob}/*` }))+ )+ .sort(+ (a, b) =>+ literalSegments(b.url) - literalSegments(a.url) ||+ segmentCount(b.url) - segmentCount(a.url)+ );+}diff --git a/frontend/asset-processor/route-map-parser.test.mjs b/frontend/asset-processor/route-map-parser.test.mjsnew file mode 100644index 00000000000..0eb0b139e9f--- /dev/null+++ b/frontend/asset-processor/route-map-parser.test.mjs@@ -0,0 +1,299 @@+import { parseAst as parse } from "rolldown/parseAst";+import { describe, expect, it } from "vitest";+import {+ buildRouteTree,+ bundleByRouteFor,+ deriveRoutes,+ parseRouteMap,+ urlTableFor,+} from "./route-map-parser";++function read(source, opts = {}) {+ return parseRouteMap(parse(source), {+ filename: "test-route-map.js",+ source,+ label: "PLUGIN test",+ ...opts,+ });+}++// `core: true` stands in for core's own maps, whose routes get no implicit bundle.+function tablesFor(sources) {+ const maps = sources.map((source) =>+ typeof source === "string"+ ? read(source)+ : { core: source.core, ...read(source.source, source) }+ );+ const derived = deriveRoutes(buildRouteTree(maps).root);++ return {+ bundleByRoute: bundleByRouteFor(derived),+ urls: urlTableFor(derived),+ };+}++describe("parseRouteMap", () => {+ it("reads a function map", () => {+ const map = read(`export default function () {+ this.route("chat", { bundleName: "chat" }, function () {+ this.route("channel", { path: "/c/:channelTitle/:channelId" });+ });+ }`);++ expect(map.resource).toBe(null);+ expect(map.routes[0].name).toBe("chat");+ expect(map.routes[0].options).toEqual({ bundleName: "chat" });+ expect(map.routes[0].children[0].options).toEqual({+ path: "/c/:channelTitle/:channelId",+ });+ });++ it("reads a resource map, ignoring `path` the router ignores", () => {+ const map = read(`export default {+ resource: "admin.adminPlugins.show",+ path: "/plugins",+ map() {+ this.route("hooks");+ },+ };`);++ expect(map.resource).toBe("admin.adminPlugins.show");+ expect(map.routes).toHaveLength(1);+ });++ it.each([+ [+ "a loop",+ `export default function () { for (const x of []) { this.route(x); } }`,+ ],+ [+ "a conditional",+ `export default function () { if (window.x) { this.route("a"); } }`,+ ],+ ["a computed name", `export default function () { this.route(NAME); }`],+ [+ "a spread",+ `export default function () { this.route("a", { ...opts }); }`,+ ],+ [+ "a non-literal option",+ `export default function () { this.route("a", { path: X }); }`,+ ],+ ["another call", `export default function () { setup(); }`],+ [+ "an arrow callback",+ `export default function () { this.route("a", () => {}); }`,+ ],+ ["no default export", `export function map() {}`],+ [+ "a `map` reference",+ `export default { resource: "a", map: someFunction };`,+ ],+ ])("rejects %s", (_label, source) => {+ expect(() => read(source)).toThrow(/test-route-map\.js:\d+:\d+/);+ });++ it("skips what it cannot read when lenient", () => {+ const map = read(+ `export default function () {+ list.forEach((x) => this.route(x));+ this.route("about");+ }`,+ { lenient: true }+ );++ expect(map.routes.map((route) => route.name)).toEqual(["about"]);+ });+});++describe("deriving", () => {+ it("nests names and paths, and resets only the name", () => {+ const { urls, bundleByRoute } = tablesFor([+ {+ core: true,+ source: `export default function () {+ this.route("admin", function () {+ this.route("adminPlugins", { path: "/plugins", resetNamespace: true }, function () {+ this.route("show", { path: "/:plugin_id" });+ });+ });+ }`,+ },+ `export default {+ resource: "admin.adminPlugins.show",+ map() {+ this.route("discourse-chat-incoming-webhooks", { path: "hooks", bundleName: "webhooks" }, function () {+ this.route("edit", { path: "/:id/edit" });+ });+ },+ };`,+ ]);++ expect(bundleByRoute).toEqual({+ "admin-plugins.show.discourse-chat-incoming-webhooks": "webhooks",+ "admin-plugins.show.discourse-chat-incoming-webhooks.edit": "webhooks",+ });+ expect(urls).toEqual([+ { bundleName: "webhooks", url: "admin/plugins/*/hooks/*" },+ ]);+ });++ it("gives a plugin route with no bundleName the default bundle", () => {+ const { bundleByRoute, urls } = tablesFor([+ {+ core: true,+ source: `export default function () { this.route("user"); }`,+ },+ `export default function () {+ this.route("standalone");+ this.route("named", { bundleName: "named" });+ }`,+ ]);++ expect(bundleByRoute).toEqual({ standalone: "default", named: "named" });+ expect(bundleByRoute.user).toBeUndefined();+ expect(urls).toContainEqual({ bundleName: "default", url: "standalone/*" });+ });++ it("groups siblings into one bundle", () => {+ const { bundleByRoute, urls } = tablesFor([+ `export default function () {+ this.route("chat", { bundleName: "chat" }, function () {+ this.route("disabled", { bundleName: "foo" });+ this.route("search", { bundleName: "foo" });+ });+ }`,+ ]);++ expect(bundleByRoute).toEqual({+ chat: "chat",+ "chat.disabled": "foo",+ "chat.search": "foo",+ });+ expect(urls).toEqual([+ { bundleName: "foo", url: "chat/disabled/*" },+ { bundleName: "foo", url: "chat/search/*" },+ { bundleName: "chat", url: "chat/*" },+ ]);+ });++ it("keeps a splat distinct from a dynamic segment", () => {+ // A `:dynamic` is one segment, a `*splat` is the rest of the route path.+ const { urls } = tablesFor([+ {+ core: true,+ source: `export default function () {+ this.route("discovery", { path: "/" }, function () {+ this.route("category", { path: "/c/*category_slug_path_with_id" });+ });+ }`,+ },+ `export default {+ resource: "discovery.category",+ map() {+ this.route("extras", { path: "/:id/extras", bundleName: "extras" });+ },+ };`,+ ]);++ expect(urls).toEqual([{ bundleName: "extras", url: "c/**/*/extras/*" }]);+ });++ it("orders urls most specific first", () => {+ const { urls } = tablesFor([+ `export default function () {+ this.route("chat", { bundleName: "chat" }, function () {+ this.route("search", { bundleName: "foo" });+ });+ }`,+ ]);++ expect(urls.map((entry) => entry.url)).toEqual(["chat/search/*", "chat/*"]);+ });++ it("takes a bundle name from the map itself", () => {+ const { bundleByRoute } = tablesFor([+ `export default {+ resource: "admin",+ bundleName: "workflows",+ map() {+ this.route("workflows", function () { this.route("show"); });+ this.route("variables");+ },+ };`,+ `export default function () { this.route("admin"); }`,+ ]);++ expect(bundleByRoute["admin.workflows"]).toBe("workflows");+ expect(bundleByRoute["admin.workflows.show"]).toBe("workflows");+ expect(bundleByRoute["admin.variables"]).toBe("workflows");+ });++ it("lets a route override the bundle the map names", () => {+ const { bundleByRoute } = tablesFor([+ `export default {+ resource: "admin",+ bundleName: "workflows",+ map() {+ this.route("a");+ this.route("b", { bundleName: "other" }, function () {+ this.route("c");+ });+ },+ };`,+ `export default function () { this.route("admin"); }`,+ ]);++ expect(bundleByRoute["admin.a"]).toBe("workflows");+ expect(bundleByRoute["admin.b"]).toBe("other");+ expect(bundleByRoute["admin.b.c"]).toBe("other");+ });++ it("rejects a computed bundle name on the map", () => {+ expect(() =>+ tablesFor([+ `export default {+ resource: "admin",+ bundleName: SOME_CONSTANT,+ map() { this.route("a"); },+ };`,+ ])+ ).toThrow(/`bundleName` must be a string literal/);+ });++ it("orders a dynamic child ahead of the parent it sits under", () => {+ const { urls } = tablesFor([+ `export default function () {+ this.route("workflows", { bundleName: "workflows" }, function () {+ this.route("show", { path: "/:id", bundleName: "editor" });+ });+ }`,+ ]);++ expect(urls.map((entry) => entry.url)).toEqual([+ "workflows/*/*",+ "workflows/*",+ ]);+ });++ it("merges maps that extend the same route", () => {+ const { bundleByRoute } = tablesFor([+ `export default function () { this.route("chat", { bundleName: "chat" }); }`,+ `export default function () {+ this.route("chat", function () { this.route("extra"); });+ }`,+ ]);++ expect(bundleByRoute["chat.extra"]).toBe("chat");+ });++ it("reports a resource that mounts on nothing", () => {+ const { unmounted } = buildRouteTree([+ read(+ `export default { resource: "discovery.filter", map() { this.route("x"); } };`+ ),+ ]);++ expect(unmounted).toHaveLength(1);+ });+});diff --git a/frontend/discourse/app/app.js b/frontend/discourse/app/app.jsindex c0ef17a0a2f..2aa1817b795 100644--- a/frontend/discourse/app/app.js+++ b/frontend/discourse/app/app.js@@ -61,13 +61,37 @@ window.moduleBroker = {},};+// `Resolver#addModules` expects the same namespacing as the eager modules.+function registerRouteBundles(bundle, prefix) {+ window._embroiderRouteBundles_ ??= [];++ for (const { names, load } of bundle.routes ?? []) {+ window._embroiderRouteBundles_.push({+ names,+ load: async () => {+ const routeModules = (await load()).default;++ return {+ default: Object.fromEntries(+ Object.entries(routeModules).map(([key, mod]) => [+ `${prefix}/${key}`,+ mod,+ ])+ ),+ };+ },+ });+ }+}+async function loadThemeFromModulePreload(link) {const themeId = link.dataset.themeId;try {- const compatModules = (await import(/* @vite-ignore */ link.href)).default;- for (const [key, mod] of Object.entries(compatModules)) {+ const bundle = await import(/* @vite-ignore */ link.href);+ for (const [key, mod] of Object.entries(bundle.compatModules)) {define(`discourse/theme-${themeId}/${key}`, () => mod);}+ registerRouteBundles(bundle, `discourse/theme-${themeId}`);} catch (error) {// eslint-disable-next-line no-consoleconsole.error(@@ -86,10 +110,11 @@ async function loadThemeFromModulePreload(link) {async function loadPluginFromModulePreload(link) {const pluginName = link.dataset.pluginName;try {- const compatModules = (await import(/* @vite-ignore */ link.href)).default;- for (const [key, mod] of Object.entries(compatModules)) {+ const bundle = await import(/* @vite-ignore */ link.href);+ for (const [key, mod] of Object.entries(bundle.compatModules)) {define(`discourse/plugins/${pluginName}/${key}`, () => mod);}+ registerRouteBundles(bundle, `discourse/plugins/${pluginName}`);} catch (error) {if (DEBUG) {if (isRailsTesting() || isTesting()) {diff --git a/frontend/discourse/app/lib/deferred-class-modifications.js b/frontend/discourse/app/lib/deferred-class-modifications.jsnew file mode 100644index 00000000000..16388304b68--- /dev/null+++ b/frontend/discourse/app/lib/deferred-class-modifications.js@@ -0,0 +1,52 @@+// A `modifyClass` call can name a class whose module has not loaded yet. Route bundles flush+// these when they arrive; a component registers itself, because nothing else registers it.++const pendingModifications = new Map();+const lazyClasses = new Map();++export function lazyClassFor(resolverName) {+ const klass = lazyClasses.get(resolverName);++ return klass ? { class: klass } : null;+}++export function deferClassModification(resolverName, pendingModification) {+ const modifications = pendingModifications.get(resolverName) ?? [];++ modifications.push(pendingModification);+ pendingModifications.set(resolverName, modifications);+}++// Appended to every plugin and theme component module by `discourse-register-components`.+export function registerModuleForModifyClass(path, klass) {+ const resolverName = `component:${path}`;++ lazyClasses.set(resolverName, klass);+ applyPending(resolverName);+}++export function resetDeferredClassModifications() {+ pendingModifications.clear();+}++// Route modules arrive as a batch through `Resolver#addModules` rather than one at a time.+export function applyDeferredClassModifications() {+ for (const resolverName of [...pendingModifications.keys()]) {+ applyPending(resolverName);+ }+}++// Taking them off first is what stops one that defers again from looping.+function applyPending(resolverName) {+ const modifications = pendingModifications.get(resolverName);++ if (!modifications) {+ return;+ }++ pendingModifications.delete(resolverName);++ for (const pendingModification of modifications) {+ pendingModification();+ }+}diff --git a/frontend/discourse/app/lib/plugin-api.gjs b/frontend/discourse/app/lib/plugin-api.gjsindex 6292957b2bb..428db530ebd 100644--- a/frontend/discourse/app/lib/plugin-api.gjs+++ b/frontend/discourse/app/lib/plugin-api.gjs@@ -67,6 +67,10 @@ import {_INTERNAL_SOURCE_KEY,CORE_SOURCE,} from "discourse/lib/customization-source";+import {+ deferClassModification,+ lazyClassFor,+} from "discourse/lib/deferred-class-modifications";import deprecated from "discourse/lib/deprecated";import { registerDesktopNotificationHandler } from "discourse/lib/desktop-notifications";import { downloadCalendar } from "discourse/lib/download-calendar";@@ -238,8 +242,7 @@ class _PluginApi {return this.container.lookup(path);}- _resolveClass(resolverName, opts) {- opts = opts || {};+ _resolveClass(resolverName) {const normalized = this.container.registry.normalize(resolverName);if (this.container.cache[normalized] ||@@ -256,18 +259,7 @@ class _PluginApi {let klass;if (!blockedModifications.includes(normalized)) {- klass = this.container.factoryFor(normalized);- }-- if (!klass) {- if (!opts.ignoreMissing) {- // eslint-disable-next-line no-console- console.warn(- consolePrefix(),- `"${normalized}" was not found by modifyClass`- );- }- return;+ klass = this.container.factoryFor(normalized) || lazyClassFor(normalized);}return klass;@@ -295,8 +287,11 @@ class _PluginApi {modifyClass(resolverName, changes, opts) {this.#deprecateModifyClass(resolverName, "modifyClass");- const klass = this._resolveClass(resolverName, opts);+ const klass = this._resolveClass(resolverName);if (!klass) {+ this.#deferModification(resolverName, () =>+ this.modifyClass(resolverName, changes, opts)+ );return;}@@ -336,8 +331,11 @@ class _PluginApi {modifyClassStatic(resolverName, changes, opts) {this.#deprecateModifyClass(resolverName, "modifyClassStatic");- const klass = this._resolveClass(resolverName, opts);+ const klass = this._resolveClass(resolverName);if (!klass) {+ this.#deferModification(resolverName, () =>+ this.modifyClassStatic(resolverName, changes, opts)+ );return;}@@ -3943,6 +3941,14 @@ class _PluginApi {_registerConditionType(ConditionClass, this.source);}+ // The module may not have been evaluated yet, and registers itself when it is.+ #deferModification(resolverName, retry) {+ deferClassModification(+ this.container.registry.normalize(resolverName),+ retry+ );+ }+#deprecateModifyClass(resolverName, apiName) {if (!resolverName.startsWith("model:")) {return;diff --git a/frontend/discourse/app/loader-shims.js b/frontend/discourse/app/loader-shims.jsindex 625742993dc..4b6caae8787 100644--- a/frontend/discourse/app/loader-shims.js+++ b/frontend/discourse/app/loader-shims.js@@ -63,6 +63,8 @@ loaderShim("@ember/template-factory", () =>importSync("@ember/template-factory"));loaderShim("@ember/template", () => importSync("@ember/template"));+// Needed in production: plugins register their dynamic imports so tests wait for them.+loaderShim("@ember/test-waiters", () => importSync("@ember/test-waiters"));loaderShim("@ember/utils", () => importSync("@ember/utils"));loaderShim("@floating-ui/dom", () => importSync("@floating-ui/dom"));loaderShim("@glimmer/component", () => importSync("@glimmer/component"));diff --git a/frontend/discourse/app/resolver.js b/frontend/discourse/app/resolver.jsindex 670a22e72be..e5917e8d5f1 100644--- a/frontend/discourse/app/resolver.js+++ b/frontend/discourse/app/resolver.js@@ -1,5 +1,6 @@import { dasherize, decamelize } from "@ember/string";import Resolver from "ember-resolver";+import { applyDeferredClassModifications } from "discourse/lib/deferred-class-modifications";import deprecated, { withSilencedDeprecations } from "discourse/lib/deprecated";import DiscourseTemplateMap from "discourse/lib/discourse-template-map";import { findHelper } from "discourse/lib/helpers";@@ -371,6 +372,12 @@ export function buildResolver(baseName) {for (const [key, value] of Object.entries(modules)) {define(key, () => value);}++ // Both are memoized, so a lazily-loaded route bundle is invisible to them until rebuilt.+ expireModuleTrieCache();+ DiscourseTemplateMap.setModuleNames(Object.keys(requirejs.entries));++ applyDeferredClassModifications();}/**diff --git a/frontend/discourse/tests/helpers/qunit-helpers.js b/frontend/discourse/tests/helpers/qunit-helpers.jsindex 4fb2425ddce..f80ee04a8d2 100644--- a/frontend/discourse/tests/helpers/qunit-helpers.js+++ b/frontend/discourse/tests/helpers/qunit-helpers.js@@ -48,6 +48,7 @@ import { resetAdditionalReportModes } from "discourse/lib/admin-report-additionaimport { rollbackAllPrepends } from "discourse/lib/class-prepend";import { _clearRegisteredActions } from "discourse/lib/composer/actions-registry";import { clearPopupMenuOptions } from "discourse/lib/composer/custom-popup-menu-options";+import { resetDeferredClassModifications } from "discourse/lib/deferred-class-modifications";import deprecated, { clearBacklog } from "discourse/lib/deprecated";import { clearDesktopNotificationHandlers } from "discourse/lib/desktop-notifications";import { visible as isVisible } from "discourse/lib/dom-utils";@@ -224,6 +225,7 @@ export function testCleanup(container, app) {resetAdminDashboardReportRenderers();resetAdminReportRelatedItemsRenderers();resetAdminDashboardSections();+ resetDeferredClassModifications();resetExtraClasses();clearOutletCache();clearHTMLCache();diff --git a/frontend/discourse/tests/unit/ember/resolver-test.js b/frontend/discourse/tests/unit/ember/resolver-test.jsindex 6e2405dc3f4..ae0202e60c5 100644--- a/frontend/discourse/tests/unit/ember/resolver-test.js+++ b/frontend/discourse/tests/unit/ember/resolver-test.js@@ -1,8 +1,12 @@import { setupTest } from "ember-qunit";import { module, test } from "qunit";+import {+ applyDeferredClassModifications,+ deferClassModification,+} from "discourse/lib/deferred-class-modifications";import { withSilencedDeprecations } from "discourse/lib/deprecated";import DiscourseTemplateMap from "discourse/lib/discourse-template-map";-import { buildResolver } from "discourse/resolver";+import { buildResolver, expireModuleTrieCache } from "discourse/resolver";import { registerTemporaryModule } from "discourse/tests/helpers/temporary-module-helper";let resolver;@@ -632,4 +636,81 @@ module("Unit | Ember | resolver", function (hooks) {"my-second-component");});++ test("addModules exposes modules which arrive after boot", function (assert) {+ const templateModule =+ "discourse/plugins/my-fake-plugin/discourse/templates/lazy-route";+ const serviceModule =+ "discourse/plugins/my-fake-plugin/discourse/services/lazy-thing";++ // Build both caches first, the way a lazily-loaded route bundle finds them.+ expireModuleTrieCache();+ DiscourseTemplateMap.setModuleNames(Object.keys(requirejs.entries));+ resolve("service:some-service-to-populate-the-trie");++ try {+ resolver.addModules({+ [templateModule]: { default: "lazy-template" },+ [serviceModule]: { default: "lazy-service" },+ });++ lookupTemplate(+ assert,+ "template:lazy-route",+ "lazy-template",+ "finds a template added after the template map was built"+ );+ assert.strictEqual(+ resolve("service:lazy-thing"),+ "lazy-service",+ "finds a service added after the suffix trie was built"+ );+ } finally {+ delete requirejs.entries[templateModule];+ delete requirejs.entries[serviceModule];+ expireModuleTrieCache();+ DiscourseTemplateMap.setModuleNames(Object.keys(requirejs.entries));+ }+ });++ test("addModules applies modifications deferred for a route bundle", function (assert) {+ const routeModule =+ "discourse/plugins/my-fake-plugin/discourse/routes/lazy-route";+ const controllerModule =+ "discourse/plugins/my-fake-plugin/discourse/controllers/lazy-route";++ expireModuleTrieCache();+ resolve("service:some-service-to-populate-the-trie");++ let applied = [];+ deferClassModification("route:lazy-route", () => applied.push("route"));+ deferClassModification("controller:lazy-route", () =>+ applied.push("controller")+ );++ assert.deepEqual(applied, [], "waits for the bundle");++ try {+ resolver.addModules({+ [routeModule]: { default: "lazy-route-class" },+ [controllerModule]: { default: "lazy-controller-class" },+ });++ assert.deepEqual(+ applied.sort(),+ ["controller", "route"],+ "applies both once the bundle arrives"+ );+ assert.strictEqual(resolve("route:lazy-route"), "lazy-route-class");+ assert.strictEqual(+ resolve("controller:lazy-route"),+ "lazy-controller-class"+ );+ } finally {+ delete requirejs.entries[routeModule];+ delete requirejs.entries[controllerModule];+ expireModuleTrieCache();+ applyDeferredClassModifications();+ }+ });});diff --git a/frontend/discourse/tests/unit/lib/deferred-class-modifications-test.js b/frontend/discourse/tests/unit/lib/deferred-class-modifications-test.jsnew file mode 100644index 00000000000..5d74176e349--- /dev/null+++ b/frontend/discourse/tests/unit/lib/deferred-class-modifications-test.js@@ -0,0 +1,60 @@+import { setupTest } from "ember-qunit";+import { module, test } from "qunit";+import {+ applyDeferredClassModifications,+ deferClassModification,+ lazyClassFor,+ registerModuleForModifyClass,+ resetDeferredClassModifications,+} from "discourse/lib/deferred-class-modifications";++module("Unit | Lib | deferred-class-modifications", function (hooks) {+ setupTest(hooks);++ hooks.afterEach(function () {+ resetDeferredClassModifications();+ });++ test("keys a component by the name the resolver asks for", function (assert) {+ class Icon {}+ registerModuleForModifyClass("chat/header/icon", Icon);++ assert.strictEqual(lazyClassFor("component:chat/header/icon").class, Icon);+ assert.strictEqual(lazyClassFor("component:missing"), null);+ });++ test("applies a modification that arrived before the module", function (assert) {+ class Header {}+ let applied = 0;++ deferClassModification("component:chat-header", () => applied++);+ assert.strictEqual(applied, 0, "waits for the module");++ registerModuleForModifyClass("chat-header", Header);+ assert.strictEqual(applied, 1);++ registerModuleForModifyClass("chat-header", Header);+ assert.strictEqual(applied, 1);+ });++ test("drops a modification whose module never arrives", function (assert) {+ let applied = 0;+ deferClassModification("component:never-loaded", () => applied++);++ resetDeferredClassModifications();+ applyDeferredClassModifications();++ assert.strictEqual(applied, 0);+ });++ test("leaves modifications for other names alone", function (assert) {+ let applied = 0;+ deferClassModification("component:not-this-one", () => applied++);++ registerModuleForModifyClass("chat-header", class {});+ assert.strictEqual(applied, 0);++ applyDeferredClassModifications();+ assert.strictEqual(applied, 1);+ });+});diff --git a/lib/asset_processor.rb b/lib/asset_processor.rbindex 9dfe17f9ee4..065495e3a35 100644--- a/lib/asset_processor.rb+++ b/lib/asset_processor.rb@@ -1,7 +1,7 @@# frozen_string_literal: trueclass AssetProcessor- BASE_COMPILER_VERSION = 114+ BASE_COMPILER_VERSION = 119BUNDLE =PrecompiledBundle.new(diff --git a/lib/discourse.rb b/lib/discourse.rbindex c9ad6a8aba3..c0387760b17 100644--- a/lib/discourse.rb+++ b/lib/discourse.rb@@ -495,6 +495,8 @@ module Discourseplugins = apply_asset_filters(plugins, :js, args[:request])+ request_path = args[:request]&.path&.delete_prefix(Discourse.base_path)&.delete_prefix("/")+assets = []plugins.each do |plugin|@@ -508,6 +510,8 @@ module Discourseimportmap_name: "discourse/plugins/#{plugin.name}",external_plugin_imports:Plugin::JsManager.external_plugin_imports(plugin.directory_name, "main"),+ route_bundle:+ Plugin::JsManager.route_bundle_for_path(plugin.directory_name, "main", request_path),}endend@@ -530,6 +534,8 @@ module Discoursetype_module: true,external_plugin_imports:Plugin::JsManager.external_plugin_imports(plugin.directory_name, "admin"),+ route_bundle:+ Plugin::JsManager.route_bundle_for_path(plugin.directory_name, "admin", request_path),}endenddiff --git a/lib/plugin/js_compiler.rb b/lib/plugin/js_compiler.rbindex 5c15dbc399a..07464899f21 100644--- a/lib/plugin/js_compiler.rb+++ b/lib/plugin/js_compiler.rb@@ -7,7 +7,8 @@ class Plugin::JsCompilertree: {},entrypoints: {},filename_prefix: nil,- filename_suffix: nil+ filename_suffix: nil,+ frontend_config: nil)@plugin_name = plugin_name@tree = tree@@ -15,6 +16,7 @@ class Plugin::JsCompiler@minify = minify@filename_prefix = filename_prefix@filename_suffix = filename_suffix+ @frontend_config = frontend_configenddef compile!@@ -26,6 +28,7 @@ class Plugin::JsCompilerentrypoints: @entrypoints,filenamePrefix: @filename_prefix,filenameSuffix: @filename_suffix,+ frontendConfig: @frontend_config,},)rescue AssetProcessor::TimeoutError => ediff --git a/lib/plugin/js_manager.rb b/lib/plugin/js_manager.rbindex 0ab91a992df..e3c432db84c 100644--- a/lib/plugin/js_manager.rb+++ b/lib/plugin/js_manager.rb@@ -7,6 +7,14 @@ module Pluginattribute :request_cacheend+ @url_glob_patterns = {}++ # A plugin route map can mount on a core route, so the build needs these to derive its urls.+ CORE_ROUTE_MAPS = {+ "app-route-map.js" => "frontend/discourse/app/routes/app-route-map.js",+ "admin-route-map.js" => "frontend/discourse/admin/routes/admin-route-map.js",+ }.freeze+def self.optional_plugin_stub"data:text/javascript,/* autogenerated missing optional plugin stub */const m=new Proxy({},{get:()=>null});export default new Proxy({},{get:()=>m});"end@@ -47,21 +55,60 @@ module Pluginendend+ def self.logical_path(file_name)+ "js/plugins/#{file_name.delete_suffix(".js")}"+ end+def self.digested_logical_path_for(plugin_directory_name, entrypoint_name)manifest_entry = read_manifest(plugin_directory_name)[entrypoint_name]- "js/plugins/#{manifest_entry["fileName"].delete_suffix(".js")}" if manifest_entry+ logical_path(manifest_entry["fileName"]) if manifest_entryenddef self.import_paths_for(plugin_directory_name, entrypoint_name)- read_manifest(plugin_directory_name)[entrypoint_name]["imports"].map do- "js/plugins/#{it.delete_suffix(".js")}"- end+ read_manifest(plugin_directory_name)[entrypoint_name]["imports"].map { logical_path(it) }enddef self.external_plugin_imports(plugin_directory_name, entrypoint_name)read_manifest(plugin_directory_name)[entrypoint_name]["externalPluginImports"]end+ # The first match wins, so the build emits the most specific glob first.+ def self.route_bundle_for_path(plugin_directory_name, entrypoint_name, path)+ return if path.nil?++ bundles = read_manifest(plugin_directory_name).dig(entrypoint_name, "routeBundles")++ bundle = bundles.to_a.find { |c| path.match?(url_glob_pattern(c["url"])) }++ logical_path(bundle["fileName"]) if bundle+ end++ # Every url the build emits ends in `/*`, so a bundle covers its own route and everything+ # beneath it. Within the rest, `*` is one segment and `**` is one or more, standing in for a+ # splat route segment. A pattern only depends on its glob, so it never goes stale.+ def self.url_glob_pattern(glob)+ @url_glob_patterns[glob] ||= begin+ pattern =+ glob+ .delete_suffix("/*")+ .split("/")+ .map do |segment|+ case segment+ when "**"+ "[^/]+(?:/[^/]+)*"+ when "*"+ "[^/]+"+ else+ Regexp.escape(segment)+ end+ end+ .join("/")++ %r{\A#{pattern}(?:/.*)?\z}+ end+ end+ private_class_method :url_glob_pattern+def compile!log "Compiling #{Discourse.plugins.count} plugins..."start = Time.now@@ -120,6 +167,17 @@ module Pluginendend+ frontend_config = plugin.about_json_metadata&.dig("frontend")++ # Only a plugin that splits its own routes derives anything from these. Read from source, so+ # plugin compilation does not wait on core's build. Being in `tree` puts them in the digest+ # below, so a core route change rebuilds those plugins and no others.+ if frontend_config&.dig("staticModules") && tree.keys.any? { it.end_with?("route-map.js") }+ CORE_ROUTE_MAPS.each do |name, source_path|+ tree["__core__/#{name}"] = File.read(Rails.root.join(source_path))+ end+ end+hex_digest =Digest::SHA1.hexdigest([@@ -129,6 +187,7 @@ module PluginAssetProcessor.ember_version,minify?.to_s,plugin.name,+ frontend_config.to_json,].join,)base36_digest = hex_digest.to_i(16).to_s(36).first(8)@@ -155,6 +214,7 @@ module Pluginentrypoints: entrypoints_config,filename_prefix:,filename_suffix:,+ frontend_config:,)result = compiler.compile!@@ -174,6 +234,7 @@ module PluginfileName: file_name,imports: info["imports"],externalPluginImports: info["externalPluginImports"],+ routeBundles: info["routeBundles"],}endenddiff --git a/spec/lib/asset_processor_spec.rb b/spec/lib/asset_processor_spec.rbindex a1a1428dcf6..74614d9aea9 100644--- a/spec/lib/asset_processor_spec.rb+++ b/spec/lib/asset_processor_spec.rb@@ -478,8 +478,9 @@ RSpec.describe AssetProcessor doexpect(entrypoint(result, "main")["code"]).to include("setComponentTemplate")expect(entrypoint(result, "main")["code"]).to include(- "bar = setComponentTemplate(__COLOCATED_TEMPLATE__, templateOnly());",+ "= setComponentTemplate(__COLOCATED_TEMPLATE__, templateOnly());",)+ expect(entrypoint(result, "main")["code"]).to include('registerModuleForModifyClass("bar",')endit "handles colocation of connectors" dodiff --git a/spec/lib/plugin/js_manager_spec.rb b/spec/lib/plugin/js_manager_spec.rbnew file mode 100644index 00000000000..ddb726fcdb8--- /dev/null+++ b/spec/lib/plugin/js_manager_spec.rb@@ -0,0 +1,76 @@+# frozen_string_literal: true++RSpec.describe Plugin::JsManager do+ describe ".route_bundle_for_path" do+ let(:manifest) do+ {+ "main" => {+ "routeBundles" => [+ { "url" => "chat/visualizer/*", "fileName" => "visualizer.js" },+ { "url" => "chat/*", "fileName" => "chat.js" },+ { "url" => "u/*/preferences/chat/*", "fileName" => "preferences.js" },+ ],+ },+ }+ end++ before { allow(described_class).to receive(:read_manifest).and_return(manifest) }++ def bundle_for(path)+ described_class.route_bundle_for_path("chat", "main", path)+ end++ it "matches an exact url" do+ expect(bundle_for("chat/visualizer")).to eq("js/plugins/visualizer")+ end++ it "prefers the first matching glob, so a specific url wins over a wildcard covering it" do+ # `chat/visualizer` also matches `chat/*`, but is declared first.+ expect(bundle_for("chat/visualizer")).to eq("js/plugins/visualizer")+ expect(bundle_for("chat/c/some-channel/1")).to eq("js/plugins/chat")+ end++ it "matches the bare parent of a trailing wildcard" do+ expect(bundle_for("chat")).to eq("js/plugins/chat")+ end++ it "does not let a wildcard match a sibling with the same prefix" do+ expect(bundle_for("chatter")).to eq(nil)+ end++ it "matches a wildcard standing in for a dynamic segment" do+ expect(bundle_for("u/bob/preferences/chat")).to eq("js/plugins/preferences")+ end++ it "requires the segments after a wildcard to match" do+ expect(bundle_for("u/bob/preferences/account")).to eq(nil)+ expect(bundle_for("u/bob/preferences")).to eq(nil)+ end++ context "with a splat segment" do+ let(:manifest) do+ { "main" => { "routeBundles" => [{ "url" => "c/**/edit/*", "fileName" => "edit.js" }] } }+ end++ it "matches one segment or many, because a splat eats the rest of the route path" do+ expect(bundle_for("c/announcements/edit")).to eq("js/plugins/edit")+ expect(bundle_for("c/parent/child/5/edit")).to eq("js/plugins/edit")+ end++ it "still requires the segments around it to match" do+ expect(bundle_for("c/announcements/5")).to eq(nil)+ expect(bundle_for("c/edit")).to eq(nil)+ expect(bundle_for("x/announcements/5/edit")).to eq(nil)+ end+ end++ it "returns nothing for an unrelated path" do+ expect(bundle_for("latest")).to eq(nil)+ end++ it "returns nothing when the plugin splits no routes" do+ allow(described_class).to receive(:read_manifest).and_return({ "main" => {} })+ expect(bundle_for("chat")).to eq(nil)+ end+ end+end