The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Stop pretending the cache is up to date (#76898)
metamben5c0455b3ecfd
diff --git a/src/metabase/query_processor/middleware/cache_backend/db.clj b/src/metabase/query_processor/middleware/cache_backend/db.cljindex daa49327a98..4ce23d378a8 100644--- a/src/metabase/query_processor/middleware/cache_backend/db.clj+++ b/src/metabase/query_processor/middleware/cache_backend/db.clj@@ -84,15 +84,15 @@Updates the raw table, not the `:model/QueryCache` model: the model's `:hook/updated-at-timestamped?` before-updatehook makes toucan2 run a non-atomic SELECT-then-UPDATE-by-PK, which would move the conditional lease check into the- SELECT and let two processes both win. The raw table keeps this a single atomic conditional UPDATE; we set- `updated_at` ourselves in place of the hook."+ SELECT and let two processes both win. The raw table keeps this a single atomic conditional UPDATE, which also means+ the hook does not run -- so `updated_at` is left alone. That is intentional: `updated_at` means \"when the results+ blob was last written\", read that way by [[cache-fresh?]], [[purge-old-cache-entries!]], and the EE refresh+ scheduler; bumping it here would let a crashed refresh silently extend the row's freshness (#76856)."[query-hash lease-ms]- (let [now (t/offset-date-time)]- (pos? (t2/update! (t2/table-name :model/QueryCache)- {:query_hash query-hash- [:coalesce :refresh_started_at lease-free-sentinel] [:< (ms-ago lease-ms)]}- {:refresh_started_at now- :updated_at now}))))+ (pos? (t2/update! (t2/table-name :model/QueryCache)+ {:query_hash query-hash+ [:coalesce :refresh_started_at lease-free-sentinel] [:< (ms-ago lease-ms)]}+ {:refresh_started_at (t/offset-date-time)})))(defn- purge-old-cache-entries!"Delete any cache entries that are older than the global max age `max-cache-entry-age-seconds` (currently 3 months)."diff --git a/test/metabase/query_processor/middleware/cache_test.clj b/test/metabase/query_processor/middleware/cache_test.cljindex a3d12fc3573..520dec7ecb4 100644--- a/test/metabase/query_processor/middleware/cache_test.clj+++ b/test/metabase/query_processor/middleware/cache_test.clj@@ -246,6 +246,24 @@(testing "an abandoned lease (older than the caller's tolerance) can be taken over"(is (true? (backend.db/try-acquire-refresh-lease! query-hash -1)))))))+(deftest refresh-lease-does-not-bump-updated-at-test+ (testing (str "try-acquire-refresh-lease! must not touch updated_at (regression for #76856). "+ "updated_at means 'when the results blob was last written' -- freshness, purging, and the EE refresh "+ "scheduler all read it that way. Bumping it on lease claim makes a crashed refresh look freshly "+ "written, so the stale row is treated as fresh for a full additional cache window.")+ #_{:clj-kondo/ignore [:discouraged-var]}+ (let [original-updated-at (t/offset-date-time "2020-01-01T00:00Z")]+ (mt/with-temp [:model/QueryCache {query-hash :query_hash} {:query_hash (byte-array (range 32))+ :results (byte-array [0])+ :updated_at original-updated-at}]+ (is (true? (backend.db/try-acquire-refresh-lease! query-hash (u/minutes->ms 5))))+ (let [{:keys [updated_at refresh_started_at]}+ (t2/select-one :model/QueryCache :query_hash query-hash)]+ (testing "the lease was claimed"+ (is (some? refresh_started_at)))+ (testing "updated_at was left alone -- results have not actually been rewritten yet"+ (is (= (t/instant original-updated-at) (t/instant updated_at)))))))))+(deftest stale-while-revalidate-test(testing "an expired entry whose refresh lease is already held by another process is served stale instead ofrecomputed, so concurrent requests don't stampede the data warehouse"