The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
GHY-4215 Ignore invalidated-at when checking for early refresh (#78945)
Ben Grabowbb4fa3fcf4d6
diff --git a/src/metabase/query_processor/middleware/cache.clj b/src/metabase/query_processor/middleware/cache.cljindex f013e8a3347..b5e04124c66 100644--- a/src/metabase/query_processor/middleware/cache.clj+++ b/src/metabase/query_processor/middleware/cache.clj@@ -236,13 +236,14 @@A fraction rather than a fixed lead time so it scales with each strategy's own duration: a lead time longer thanthe duration would put an entry in the refresh zone the moment it was written. Returns false when `window-ms` isnil, since a strategy whose boundary doesn't slide (`:schedule`) has no \"about to expire\"."- [updated-at invalidated-at window-ms early-refresh-ratio]+ [updated-at window-ms early-refresh-ratio](boolean (and updated-atwindow-ms- ;; how much of the window is left: the boundary slides with the clock, so the distance from it to- ;; `updated-at` is the time until this entry falls behind it- (let [remaining-ms (.toMillis (t/duration (t/instant invalidated-at) (t/instant updated-at)))]- (< remaining-ms (* early-refresh-ratio window-ms))))))+ ;; how much of the window is left, measured from the clock rather than from the strategy's+ ;; `invalidated-at`: an explicit invalidation pins that boundary to a fixed instant, and the distance+ ;; from a pinned boundary is the entry's age since the invalidation, not its remaining freshness+ (let [age-ms (.toMillis (t/duration (t/instant updated-at) (t/instant)))]+ (< (- window-ms age-ms) (* early-refresh-ratio window-ms))))))(defn- not-too-stale?"Whether an entry last written at `updated-at` is close enough to `invalidated-at` (the strategy's freshness@@ -294,7 +295,6 @@(and (cache-fresh? updated-at invalidated-at)(due-for-early-refresh?updated-at- invalidated-at(fresh-duration-ms strategy)(cache/query-caching-early-refresh-ratio))(i/try-acquire-refresh-lease! *backend* query-hash *refresh-lease-duration-ms*))diff --git a/test/metabase/query_processor/middleware/cache_test.clj b/test/metabase/query_processor/middleware/cache_test.cljindex bd9fdfc9299..6c6bfc4bef3 100644--- a/test/metabase/query_processor/middleware/cache_test.clj+++ b/test/metabase/query_processor/middleware/cache_test.clj@@ -374,6 +374,16 @@(is (= :cached(run-query))"only the lease winner pays for the early refresh")))))+ (testing "an entry written just after an explicit invalidation has its whole window left, not the sliver since+ the invalidation"+ (with-mock-cache! [save-chan]+ (let [strategy (assoc (ttl-strategy) :invalidated-at (t/offset-date-time t0))]+ (mt/with-clock (t/plus t0 (t/seconds 1))+ (run-query :cache-strategy strategy)+ (mt/wait-for-result save-chan))+ (mt/with-clock (t/plus t0 (t/seconds 31))+ (is (= :cached+ (run-query :cache-strategy strategy)))))))(testing "early refreshing is off when the ratio is 0"(with-mock-cache! [save-chan](mt/with-temporary-setting-values [query-caching-early-refresh-ratio 0.0]