Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

GHY-4215 Ignore invalidated-at when checking for early refresh (#78945)

Ben Grabowbb4fa3fcf4d6
diff --git a/src/metabase/query_processor/middleware/cache.clj b/src/metabase/query_processor/middleware/cache.clj
index f013e8a3347..b5e04124c66 100644
--- a/src/metabase/query_processor/middleware/cache.clj
+++ b/src/metabase/query_processor/middleware/cache.clj
@@ -236,13 +236,14 @@
A fraction rather than a fixed lead time so it scales with each strategy's own duration: a lead time longer than
the duration would put an entry in the refresh zone the moment it was written. Returns false when `window-ms` is
nil, since a strategy whose boundary doesn't slide (`:schedule`) has no \"about to expire\"."
- [updated-at invalidated-at window-ms early-refresh-ratio]
+ [updated-at window-ms early-refresh-ratio]
(boolean (and updated-at
window-ms
- ;; how much of the window is left: the boundary slides with the clock, so the distance from it to
- ;; `updated-at` is the time until this entry falls behind it
- (let [remaining-ms (.toMillis (t/duration (t/instant invalidated-at) (t/instant updated-at)))]
- (< remaining-ms (* early-refresh-ratio window-ms))))))
+ ;; how much of the window is left, measured from the clock rather than from the strategy's
+ ;; `invalidated-at`: an explicit invalidation pins that boundary to a fixed instant, and the distance
+ ;; from a pinned boundary is the entry's age since the invalidation, not its remaining freshness
+ (let [age-ms (.toMillis (t/duration (t/instant updated-at) (t/instant)))]
+ (< (- window-ms age-ms) (* early-refresh-ratio window-ms))))))
(defn- not-too-stale?
"Whether an entry last written at `updated-at` is close enough to `invalidated-at` (the strategy's freshness
@@ -294,7 +295,6 @@
(and (cache-fresh? updated-at invalidated-at)
(due-for-early-refresh?
updated-at
- invalidated-at
(fresh-duration-ms strategy)
(cache/query-caching-early-refresh-ratio))
(i/try-acquire-refresh-lease! *backend* query-hash *refresh-lease-duration-ms*))
diff --git a/test/metabase/query_processor/middleware/cache_test.clj b/test/metabase/query_processor/middleware/cache_test.clj
index bd9fdfc9299..6c6bfc4bef3 100644
--- a/test/metabase/query_processor/middleware/cache_test.clj
+++ b/test/metabase/query_processor/middleware/cache_test.clj
@@ -374,6 +374,16 @@
(is (= :cached
(run-query))
"only the lease winner pays for the early refresh")))))
+ (testing "an entry written just after an explicit invalidation has its whole window left, not the sliver since
+ the invalidation"
+ (with-mock-cache! [save-chan]
+ (let [strategy (assoc (ttl-strategy) :invalidated-at (t/offset-date-time t0))]
+ (mt/with-clock (t/plus t0 (t/seconds 1))
+ (run-query :cache-strategy strategy)
+ (mt/wait-for-result save-chan))
+ (mt/with-clock (t/plus t0 (t/seconds 31))
+ (is (= :cached
+ (run-query :cache-strategy strategy)))))))
(testing "early refreshing is off when the ratio is 0"
(with-mock-cache! [save-chan]
(mt/with-temporary-setting-values [query-caching-early-refresh-ratio 0.0]