The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Serialize static-viz GraalJS context creation (GHY-4077) (#77225)
dpsuttonc09cdbf49cea
diff --git a/src/metabase/channel/render/js/pool.clj b/src/metabase/channel/render/js/pool.cljindex aaa35056b93..5a2d074afff 100644--- a/src/metabase/channel/render/js/pool.clj+++ b/src/metabase/channel/render/js/pool.clj@@ -3,11 +3,29 @@(`metabase.channel.render.js.svg`) and table-cell color selection (`metabase.channel.render.js.color`). Everypooled context has the static-viz bundle, the static-viz interface, and the color-selector sources loaded.- All contexts share the process-lifetime engine and evaluate the same `Source` instances (one `defonce` per- resource), so the engine's code cache — which is keyed on the `Source` instance — holds ONE parsed copy of the- static-viz bundle (~130 MB). Building a fresh `Source` per context (the previous behavior) made the engine retain a- separate parsed copy for every pool generation, and made each generation reparse the 27 MB bundle: a- multi-hundred-MB allocation spike every context recycle.+ ### Memory model (GraalJS 25, interpreter mode — measured, see GHY-4077)++ - The engine plus the parsed sources cost ~220 MB retained, steady, paid once per process. All contexts share the+ process-lifetime engine and evaluate the same `Source` instances (one `defonce` per resource), so the engine's+ code cache — which is keyed on the `Source` instance — holds ONE parsed copy of the static-viz bundle. Building a+ fresh `Source` per context (the previous behavior) made the engine retain a separate parsed copy for every pool+ generation, and made each generation reparse the 27 MB bundle: a multi-hundred-MB allocation spike every context+ recycle.+ - Evaluating the sources in a fresh context (realm setup against the warm code cache) churns through ~500 MB of+ allocation in ~1 s. That is collectable garbage, not footprint: each extra live context retains only ~16 MB. The+ very first context in the JVM additionally pays the cold parse — ~1.2 GB / ~5 s.+ - A render itself allocates on the order of ~1 MB per row of chart data, also transient.++ ### Why context creation is serialized++ Creation being ~500 MB of churn a pop, letting the pool grow 1→3 concurrently during a render burst stacks+ ~1–1.5 GB of transient allocation on top of the in-flight renders — on a 2 GB heap (the Cloud default) that+ outruns the collector and drives GC thrash / OOM. So creation takes a private monitor+ (see [[context-creation-lock]]). A time-based throttle (\"no two creations within X seconds\") is the wrong shape:+ a burst of concurrent renders genuinely needs multiple contexts, so we let the pool grow to meet demand — it just+ warms one context per ~1 s instead of all at once, bounding the worst case to a single creation's churn. Waiting+ acquires simply queue on the pool. Pooled contexts expire after ~10 minutes with jitter so members created+ together don't all regenerate in the same window.Consumers borrow a context within [[with-static-viz-context]] and return plain data — never the context or acontext-bound `Value`. Never acquire a context while already holding one: with a small pool, nested acquires can@@ -63,18 +81,41 @@"Dirigiste pools are keyed. The key itself is arbitrary — it just has to be the same for every pool operation.":engines)+(def ^:private context-creation-lock+ "Monitor serializing pooled context creation. Even with the shared-`Source` code cache, evaluating the bundle in a+ fresh context allocates ~500 MB, so letting the pool grow 1→3 concurrently during a render burst stacks ~1–1.5 GB+ of transient allocation on top of the in-flight renders — enough to blow a 2 GB heap. Holding creations to one at a+ time bounds the worst case to a single creation (~1 s each); waiting acquires just queue on the pool meanwhile.+ Nothing else may lock this monitor."+ (Object.))++(defn- expiry-timestamp+ "Expiry timestamp for a pooled context: now + 10 minutes ± up to 3 minutes of jitter. The jitter keeps pool members+ created in the same burst from all expiring — and serially regenerating — in the same window."+ ^long []+ (+ (System/nanoTime)+ (.toNanos TimeUnit/MINUTES 7)+ (long (rand (.toNanos TimeUnit/MINUTES 6)))))++(defn- generate-pool-entry+ "Create a `[context expiry-timestamp]` tuple for the pool. At most one context creation runs at a time (see+ [[context-creation-lock]]); acquisition/use is already exclusive per context, so this only bounds pool *growth*."+ []+ (locking context-creation-lock+ [(create-static-viz-context) (expiry-timestamp)]))+(def ^:private ^Pool static-viz-context-pool"Pool of Truffle JS contexts. They are not thread-safe, so access is exclusive from acquire to release. Generating acontext is cheap — realm setup plus top-level eval against the engine's code cache, no reparse — and per-contextmemory is a realm, not a full parsed copy of the bundle. The pool targets 100% utilization with a maximum of 3contexts (to bound memory; renders hold a context exclusively, so that is also the render concurrency), but at- least 1 context is always kept in the pool to pick up. Each pooled tuple carries an expiry timestamp so a context- is recycled after 10 minutes regardless, bounding per-context leak accumulation."+ least 1 context is always kept in the pool to pick up. Each pooled tuple carries a (jittered, ~10-minute) expiry+ timestamp so a context is recycled regardless, bounding per-context leak accumulation. Context creation is+ serialized (see [[context-creation-lock]]) so pool growth never runs multiple ~500 MB creations concurrently."(let [base-controller (Pools/utilizationController 1.0 3 3)](Pool. (reify IPool$Generator(generate [_ _]- ;; Generate a tuple of the context and the expiry timestamp.- [(create-static-viz-context) (+ (System/nanoTime) (.toNanos TimeUnit/MINUTES 10))])+ (generate-pool-entry))(destroy [_ _ [^Context ctx _expiry]];; Close the context when it's disposed from the pool (expiry/idle shrink/shutdown). Without this, each;; disposed static-viz context leaks its memory: GraalVM only releases it on `close`, not on GC.diff --git a/test/metabase/channel/render/js/pool_test.clj b/test/metabase/channel/render/js/pool_test.cljnew file mode 100644index 00000000000..95f7f3a6500--- /dev/null+++ b/test/metabase/channel/render/js/pool_test.clj@@ -0,0 +1,37 @@+(ns metabase.channel.render.js.pool-test+ (:require+ [clojure.test :refer :all]+ [metabase.channel.render.js.pool :as js.pool])+ (:import+ (java.util.concurrent TimeUnit)))++(set! *warn-on-reflection* true)++(deftest generate-pool-entry-serializes-context-creation-test+ (testing "no two static-viz context creations run concurrently (#GHY-4077)"+ (let [in-flight (atom 0)+ max-in-flight (atom 0)+ fake-context (Object.)]+ (with-redefs [js.pool/create-static-viz-context+ (fn []+ (let [n (swap! in-flight inc)]+ (swap! max-in-flight max n)+ (Thread/sleep 50)+ (swap! in-flight dec)+ fake-context))]+ (let [entries (->> (repeatedly 4 #(future (#'js.pool/generate-pool-entry)))+ doall+ (mapv deref))]+ (is (= 1 @max-in-flight))+ (testing "every entry is a [context expiry-timestamp] tuple"+ (doseq [[ctx expiry-ts] entries]+ (is (identical? fake-context ctx))+ (is (int? expiry-ts)))))))))++(deftest expiry-timestamp-jitter-test+ (testing "expiry is ~10 minutes out with ±3 minutes of jitter"+ (let [now (System/nanoTime)+ expiries (repeatedly 100 #(#'js.pool/expiry-timestamp))]+ (doseq [expiry expiries+ :let [delta (- expiry now)]]+ (is (<= (.toNanos TimeUnit/MINUTES 7) delta (.toNanos TimeUnit/MINUTES 14)))))))