The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Record is_impersonated/is_db_routed correctly on failed queries (#73234)
bryan35e8509ac8ed
diff --git a/enterprise/backend/test/metabase_enterprise/impersonation/query_execution_test.clj b/enterprise/backend/test/metabase_enterprise/impersonation/query_execution_test.cljindex 9fd6b156af2..74b4bbda1a8 100644--- a/enterprise/backend/test/metabase_enterprise/impersonation/query_execution_test.clj+++ b/enterprise/backend/test/metabase_enterprise/impersonation/query_execution_test.clj@@ -6,6 +6,8 @@[clojure.test :refer :all][metabase-enterprise.impersonation.util-test :as impersonation.util-test][metabase.query-processor :as qp]+ [metabase.query-processor.error-type :as qp.error-type]+ [metabase.query-processor.pipeline :as qp.pipeline][metabase.query-processor.util :as qp.util][metabase.test :as mt][toucan2.core :as t2]))@@ -30,3 +32,24 @@(mt/with-test-user :rasta(qp/process-query (qp/userland-query (mt/mbql-query venues {:limit 1}) {:context :question}))(is (false? (:is_impersonated (latest-query-execution))))))))++(deftest is-impersonated-true-when-impersonated-query-fails-test+ (testing "When an impersonated query throws during execution, the persisted QueryExecution row should still+ record is_impersonated=true. Reproduces a Clojure-binding gotcha: by the time the catch block in+ process-userland-query-middleware fires, the impersonation `binding` established by the EE postprocessing+ middleware has already been popped during stack unwind."+ (mt/with-premium-features #{:advanced-permissions}+ (mt/with-model-cleanup [:model/QueryExecution]+ (binding [qp.util/*execute-async?* false+ qp.pipeline/*run* (fn [_query _rff]+ (throw (ex-info "Boom" {:type qp.error-type/qp})))]+ (impersonation.util-test/with-impersonations!+ {:impersonations [{:db-id (mt/id) :attribute "impersonation_attr"}]+ :attributes {"impersonation_attr" "impersonation_role"}}+ (try+ (qp/process-query (qp/userland-query (mt/mbql-query venues {:limit 1}) {:context :question}))+ (catch Throwable _))+ (is (true? (:is_impersonated (latest-query-execution)))+ "QueryExecution row for a FAILED impersonated query should still record is_impersonated=true")+ (is (some? (:error (latest-query-execution)))+ "Sanity check: the QueryExecution row should have an error message")))))))diff --git a/src/metabase/query_processor/execute.clj b/src/metabase/query_processor/execute.cljindex df4ae9aa5dd..08ccc9b7caa 100644--- a/src/metabase/query_processor/execute.clj+++ b/src/metabase/query_processor/execute.clj@@ -5,6 +5,7 @@[metabase.query-processor.middleware.cache :as cache][metabase.query-processor.middleware.enterprise :as qp.middleware.enterprise][metabase.query-processor.middleware.permissions :as qp.perms]+ [metabase.query-processor.middleware.process-userland-query :as qp.process-userland-query][metabase.query-processor.middleware.update-used-cards :as update-used-cards][metabase.query-processor.pipeline :as qp.pipeline][metabase.query-processor.schema :as qp.schema]@@ -48,7 +49,11 @@(f (f query rff)) -> (f query rff)All of these middlewares assume MBQL 5."- [#'qp.middleware.enterprise/swap-destination-db-middleware+ ;; Must be FIRST in this list so it runs INNERMOST (the LAST element of `reduce` ends up running FIRST). It needs+ ;; to execute INSIDE the `binding` blocks established by the EE postprocessing middlewares below. See+ ;; [[metabase.query-processor.middleware.process-userland-query/capture-execution-context-middleware]] for why.+ [#'qp.process-userland-query/capture-execution-context-middleware+ #'qp.middleware.enterprise/swap-destination-db-middleware#'qp.middleware.enterprise/apply-impersonation-postprocessing-middleware#'update-used-cards/update-used-cards!#'add-native-form-to-result-metadatadiff --git a/src/metabase/query_processor/middleware/process_userland_query.clj b/src/metabase/query_processor/middleware/process_userland_query.cljindex 3c5e96b55c7..387ed0e094d 100644--- a/src/metabase/query_processor/middleware/process_userland_query.clj+++ b/src/metabase/query_processor/middleware/process_userland_query.clj@@ -175,17 +175,50 @@:result_rows 0:start_time_millis (System/currentTimeMillis)}))-(defn- enrich-with-execution-context- "Adds fields derived from postprocessing-middleware dynamic vars (`*impersonation-role*`,- `*destination-database-id*`) to `execution-info`. Must be called from inside the postprocessing rff, before the- async dispatch in [[save-execution-metadata!]] drops dynamic bindings."- [execution-info]+(defn- snapshot-execution-context+ "Reads the postprocessing-middleware dynamic vars (`*impersonation-role*`, `*destination-database-id*`) and+ returns a map of fields to merge into the QueryExecution row."+ [](let [destination-db-id (qp.middleware.enterprise/currently-destination-database-id)]- (cond-> (assoc execution-info- :is_impersonated (qp.middleware.enterprise/currently-impersonated?)- :is_db_routed (qp.middleware.enterprise/currently-db-routed?))+ (cond-> {:is_impersonated (qp.middleware.enterprise/currently-impersonated?)+ :is_db_routed (qp.middleware.enterprise/currently-db-routed?)}destination-db-id (assoc :database_id destination-db-id))))+(def ^:dynamic ^:private *execution-context-ref*+ "Bound to an atom by [[process-userland-query-middleware]] for each userland query.+ [[capture-execution-context-middleware]] writes the snapshotted impersonation/db-routing context here while the+ EE postprocessing `binding` blocks are still on the stack. The around middleware reads it in both the+ success-path rff and the error-path catch — the latter is the whole point, since by the time control reaches+ that catch the EE bindings have already been popped during exception unwind.++ Why a shared mutable ref (atom) instead of just `set!` on the dynamic var: the QP can run on a thread spawned+ by the streaming response (via `bound-fn` / `binding-conveyor-fn`), and conveyed bindings on that thread are a+ snapshot — `set!` would mutate only the spawned thread's frame, not the caller's. A shared mutable cell that+ any thread holding a reference can read/write makes the value flow back to the caller correctly."+ nil)++(defn capture-execution-context-middleware+ "Execute middleware. Snapshots impersonation/db-routing dynamic-var values into [[*execution-context-ref*]] (if+ bound) and forwards. Must be positioned FIRST in the execute middleware list so it runs INNERMOST — i.e. inside+ the `binding` blocks established by `swap-destination-db-middleware` and+ `apply-impersonation-postprocessing-middleware`. See the comment block in+ [[metabase.query-processor.execute/middleware]] explaining the reduce order."+ [qp]+ (fn [query rff]+ (when *execution-context-ref*+ (reset! *execution-context-ref* (snapshot-execution-context)))+ (qp query rff)))++(defn- enrich-with-execution-context+ "Merges the snapshotted execution context (from [[*execution-context-ref*]]) into `execution-info`. Always+ includes `:is_impersonated` and `:is_db_routed` (defaulting to `false`) so the QueryExecution row has a stable+ shape even when the snapshot didn't run (e.g. tests that bypass [[capture-execution-context-middleware]])."+ [execution-info]+ (merge {:is_impersonated false+ :is_db_routed false}+ execution-info+ (some-> *execution-context-ref* deref)))+(mu/defn process-userland-query-middleware :- ::qp.schema/qp"Around middleware.@@ -206,27 +239,27 @@(analytics/set-gauge! :metabase.query-processor/computed-weak-map-queries (lib.computed/weak-map-population))(if-not (qp.util/userland-query? query)(qp query rff)- (let [query (assoc-in query [:info :query-hash] (qp.util/query-hash query))- execution-info (query-execution-info query)]- (letfn [(rff* [metadata]- (let [;; we only need the preprocessed query to find field usages, so make sure we don't return it- result (rff (dissoc metadata :preprocessed_query))- ;; Enrich here, while the impersonation / db-routing dynamic var bindings established by- ;; postprocessing middleware are still in effect. Doing this at the top of the around- ;; middleware was wrong — preprocessing hadn't run yet and the bindings weren't set.- execution-info (enrich-with-execution-context execution-info)]- (add-and-save-execution-metadata-xform! execution-info result)))]- (try- (qp query rff*)- (catch Throwable e- ;; Same enrichment for the error path: we're still inside `(qp query rff*)` so the postprocessing- ;; bindings are in effect.- (let [execution-info (enrich-with-execution-context execution-info)]- (save-failed-query-execution!- execution-info- (or- (some-> e ex-cause ex-message)- (ex-message e)))- (throw (ex-info (ex-message e)- {:query-execution execution-info}- e))))))))))+ ;; The atom is written from inside [[capture-execution-context-middleware]] (positioned in the execute+ ;; chain inside the EE postprocessing bindings) and read in both the success-path rff and the error-path+ ;; catch below. Reading the EE dynamic vars directly from the catch block does NOT work — Clojure pops the+ ;; EE `binding` blocks during stack unwind, so by the time control reaches the catch the values are gone.+ (binding [*execution-context-ref* (atom nil)]+ (let [query (assoc-in query [:info :query-hash] (qp.util/query-hash query))+ execution-info (query-execution-info query)]+ (letfn [(rff* [metadata]+ (let [;; we only need the preprocessed query to find field usages, so make sure we don't return it+ result (rff (dissoc metadata :preprocessed_query))+ execution-info (enrich-with-execution-context execution-info)]+ (add-and-save-execution-metadata-xform! execution-info result)))]+ (try+ (qp query rff*)+ (catch Throwable e+ (let [execution-info (enrich-with-execution-context execution-info)]+ (save-failed-query-execution!+ execution-info+ (or+ (some-> e ex-cause ex-message)+ (ex-message e)))+ (throw (ex-info (ex-message e)+ {:query-execution execution-info}+ e)))))))))))