Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Keep Quartz job class names stable across moves (#83621)

Chris Truter0ea2c779d172
diff --git a/.claude/agents/modules-backend-expert.md b/.claude/agents/modules-backend-expert.md
index 9855a22aa71..c2126a01c10 100644
--- a/.claude/agents/modules-backend-expert.md
+++ b/.claude/agents/modules-backend-expert.md
@@ -62,6 +62,7 @@ Tests:
- **Cycles are not test-gated.** No test fails on a module cycle. Track them with `module-boundary-stats`: `:scc-namespace-sizes` is the metric to watch, because splitting a module can grow `:scc-module-sizes` without removing anything. Don't add a new edge into an existing strongly connected component.
- **`.core` is a facade.** Mostly `potemkin/import-vars`, some `metabase.util.namespaces/import-fns`. Never require your own `.core` from inside the module (self-cycle). `^:dynamic` vars don't re-export usefully; export a `with-*` helper instead.
- **`.init` is eager.** Everything it requires loads at launch. Require only settings, tasks, event handlers, and multimethod registrations that must exist at startup.
+- **Keep each job's `:saved-class` when you move a `.task.*` namespace.** Quartz stores that class name in the app DB, and `task/defjob` names the class after it, not after the namespace. Don't let a bulk rename rewrite it: `metabase.app-db.quartz-test` fails if one changes.
- **Grab-bag modules** (`models`, `api`, `task`, `events`, `util`, `core.cmd`) are infrastructure. New feature code goes in a feature module named after the user-facing feature, OSS and EE sharing the name (`upload` + `enterprise/upload`).
## How to work
diff --git a/.claude/skills/add-tracing/SKILL.md b/.claude/skills/add-tracing/SKILL.md
index 82088412d5a..86caa4c22fc 100644
--- a/.claude/skills/add-tracing/SKILL.md
+++ b/.claude/skills/add-tracing/SKILL.md
@@ -275,7 +275,9 @@ When including SQL in span attributes, **always** use `tracing/best-effort-sanit
The `defjob` macro in `metabase.task.impl` automatically wraps every Quartz job with a `:tasks` root span:
```clojure
-(task/defjob ^{DisallowConcurrentExecution true} SessionCleanup [_]
+(task/defjob SessionCleanup
+ {:saved-class "metabase.session.task.session_cleanup.SessionCleanup"}
+ [_]
(cleanup-sessions!))
;; Automatically creates span: "task.SessionCleanup" {:task/name "SessionCleanup"}
```
diff --git a/.clj-kondo/config.edn b/.clj-kondo/config.edn
index c60e820d041..0e4f44fe3be 100644
--- a/.clj-kondo/config.edn
+++ b/.clj-kondo/config.edn
@@ -858,6 +858,8 @@
metabase.query-analysis.task.test-setup/with-test-setup! macros.metabase.query-analysis.task.test-setup/with-test-setup!
metabase.query-processor.streaming/streaming-response macros.metabase.query-processor.streaming/streaming-response
metabase.server.statistics-handler-test/with-server macros.metabase.server.statistics-handler-test/with-server
+ metabase.task.core/defjob-type macros.quartz/defjob-type
+ metabase.task.impl/defjob-type macros.quartz/defjob-type
metabase.test.data.users/with-group-for-user macros.metabase.test.data.users/with-group-for-user
metabase.test.util/with-temp-env-var-value! macros.metabase.test.util/with-temp-env-var-value!
metabase.test.util/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
diff --git a/.clj-kondo/macros/quartz.clj b/.clj-kondo/macros/quartz.clj
index 5628af660c0..2ec3fd53bf9 100644
--- a/.clj-kondo/macros/quartz.clj
+++ b/.clj-kondo/macros/quartz.clj
@@ -18,3 +18,8 @@
[& body]
`(let [s# ~(first body)]
(-> s# ~@(rest body))))
+
+(defmacro defjob-type
+ "A `deftype` with no fields, without the docstring and the options map."
+ [type-name & args]
+ `(deftype ~type-name [] ~@(drop-while (some-fn string? map?) args)))
diff --git a/.cljfmt.edn b/.cljfmt.edn
index 5415436824f..5c845c7d45e 100644
--- a/.cljfmt.edn
+++ b/.cljfmt.edn
@@ -113,6 +113,8 @@
defprotocol+ [[:block 1] [:inner 1]]
defrecord+ [[:block 2] [:inner 1]]
deftype+ [[:block 2] [:inner 1]]
+ metabase.task.core/defjob-type [[:inner 0] [:inner 1]]
+ metabase.task.impl/defjob-type [[:inner 0] [:inner 1]]
net.cgrand.macrovich/case [[:block 0]]
potemkin.namespaces/import-vars [[:default]]
potemkin.types/def-map-type [[:block 2] [:inner 1]]
diff --git a/CLAUDE.md b/CLAUDE.md
index ea43b812c3c..3650918e39b 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -99,6 +99,17 @@ interactive explorer with a dependency graph.
- Each `:module-exports` entry widens a nested module's visibility by one ancestor. Export every link to
make it available everywhere. OSS module `X` exports its `enterprise/X` companion automatically.
+## Moving or Renaming Quartz Jobs
+
+Keep a job's `:saved-class` the same when you move its namespace or rename its type. Quartz stores that
+class name in the app DB, and `task/defjob` gives the class exactly that name, so a move changes nothing
+that is stored.
+A new job needs one too: compile without it, and the error prints the entry to paste.
+
+Changing a `:saved-class` deletes the jobs stored under the old name, and their triggers, at the next startup.
+The tests in `metabase.app-db.quartz-test` fail with instructions when a job's class name changes, or a job
+is added.
+
## Ratchets
After changing Clojure suppressions or module-boundary escape hatches, run `./bin/mage kondo-ratchets`.
diff --git a/enterprise/backend/src/metabase_enterprise/agent_api/task/agent_api_usage_trimmer.clj b/enterprise/backend/src/metabase_enterprise/agent_api/task/agent_api_usage_trimmer.clj
index c9f8d476b5c..91af5358c9a 100644
--- a/enterprise/backend/src/metabase_enterprise/agent_api/task/agent_api_usage_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/agent_api/task/agent_api_usage_trimmer.clj
@@ -12,9 +12,7 @@
[metabase-enterprise.agent-api.db :as agent-api.db]
[metabase.metabot.settings :as metabot.settings]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -31,9 +29,11 @@
(let [calls (agent-api.db/delete-call-logs-created-before! cutoff)]
(log/infof "Agent API usage log cleanup complete. Deleted %d call rows." (or calls 0)))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete old Agent API usage log rows"}
- AgentApiUsageTrimmer [_ctx]
+(task/defjob AgentApiUsageTrimmer
+ "Delete old Agent API usage log rows"
+ {:saved-class "metabase_enterprise.agent_api.task.agent_api_usage_trimmer.AgentApiUsageTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-agent-api-usage-data!))
(defmethod task/init! ::AgentApiUsageTrimmer
diff --git a/enterprise/backend/src/metabase_enterprise/api_keys/task/api_key_usage_trimmer.clj b/enterprise/backend/src/metabase_enterprise/api_keys/task/api_key_usage_trimmer.clj
index bfb19e120a0..ff318a9666b 100644
--- a/enterprise/backend/src/metabase_enterprise/api_keys/task/api_key_usage_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/api_keys/task/api_key_usage_trimmer.clj
@@ -12,9 +12,7 @@
[metabase-enterprise.api-keys.db :as api-keys.db]
[metabase.metabot.settings :as metabot.settings]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -31,9 +29,11 @@
(let [rows (api-keys.db/delete-usage-logs-occurred-before! cutoff)]
(log/infof "API key usage log cleanup complete. Deleted %d rows." (or rows 0)))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete old API key usage log rows"}
- ApiKeyUsageTrimmer [_ctx]
+(task/defjob ApiKeyUsageTrimmer
+ "Delete old API key usage log rows"
+ {:saved-class "metabase_enterprise.api_keys.task.api_key_usage_trimmer.ApiKeyUsageTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-api-key-usage-data!))
(defmethod task/init! ::ApiKeyUsageTrimmer
diff --git a/enterprise/backend/src/metabase_enterprise/cache/task/refresh_cache_configs.clj b/enterprise/backend/src/metabase_enterprise/cache/task/refresh_cache_configs.clj
index 92017ab21fc..49e2bb2a95e 100644
--- a/enterprise/backend/src/metabase_enterprise/cache/task/refresh_cache_configs.clj
+++ b/enterprise/backend/src/metabase_enterprise/cache/task/refresh_cache_configs.clj
@@ -258,9 +258,11 @@
:schedule-refreshed schedule-refresh-count
:duration-refreshed duration-refresh-count}))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Refresh 'schedule' caches"}
- Cache [_ctx]
+(task/defjob Cache
+ "Refresh 'schedule' caches"
+ {:saved-class "metabase_enterprise.cache.task.refresh_cache_configs.Cache"
+ :concurrent? false}
+ [_ctx]
(refresh-cache-configs!))
(def ^:private cache-job
diff --git a/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/finding_trimmer.clj b/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/finding_trimmer.clj
index 056a9bc5b57..729a1d8736c 100644
--- a/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/finding_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/finding_trimmer.clj
@@ -11,9 +11,7 @@
[metabase-enterprise.content-diagnostics.settings :as cd.settings]
[metabase.task-history.core :as task-history]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -39,9 +37,11 @@
(task-history/with-task-history {:task "content-diagnostics-trimmer"}
(trim-old-findings!)))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Content Diagnostics - delete invalidated findings past retention."}
- ContentDiagnosticsFindingTrimmer [_ctx]
+(task/defjob ContentDiagnosticsFindingTrimmer
+ "Content Diagnostics - delete invalidated findings past retention."
+ {:saved-class "metabase_enterprise.content_diagnostics.task.finding_trimmer.ContentDiagnosticsFindingTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-with-history!))
(defmethod task/init! ::ContentDiagnosticsFindingTrimmer [_]
diff --git a/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/scan.clj b/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/scan.clj
index 5b038a169d0..3ac41948c20 100644
--- a/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/scan.clj
+++ b/enterprise/backend/src/metabase_enterprise/content_diagnostics/task/scan.clj
@@ -17,7 +17,7 @@
(:import
(java.time Instant)
(java.util Date)
- (org.quartz DisallowConcurrentExecution JobExecutionContext ObjectAlreadyExistsException TriggerKey)))
+ (org.quartz JobExecutionContext ObjectAlreadyExistsException TriggerKey)))
(set! *warn-on-reflection* true)
@@ -100,9 +100,11 @@
(backfill-scan!)
(scan-when-enabled!)))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Content Diagnostics - scan for problematic content."}
- ContentDiagnosticsScan [ctx]
+(task/defjob ContentDiagnosticsScan
+ "Content Diagnostics - scan for problematic content."
+ {:saved-class "metabase_enterprise.content_diagnostics.task.scan.ContentDiagnosticsScan"
+ :concurrent? false}
+ [ctx]
(scan-for-trigger! (.getKey (.getTrigger ^JobExecutionContext ctx))))
(defn- backfill-trigger []
diff --git a/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score.clj b/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score.clj
index ac00815a503..8e1df24556f 100644
--- a/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score.clj
+++ b/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score.clj
@@ -13,9 +13,7 @@
[metabase.app-db.cluster-lock :as cluster-lock]
[metabase.config.core :as config]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -222,9 +220,11 @@
:skip nil
(run-scoring! fingerprint)))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Compute and publish the Data Complexity Score."}
- DataComplexityScoring [_ctx]
+(task/defjob DataComplexityScoring
+ "Compute and publish the Data Complexity Score."
+ {:saved-class "metabase_enterprise.data_complexity_score.task.complexity_score.DataComplexityScoring"
+ :concurrent? false}
+ [_ctx]
(with-scoring-claim! {:cooldown-hours cron-cooldown-hours} run-claim!))
(defn maybe-emit-boot-score!
diff --git a/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score_trimmer.clj b/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score_trimmer.clj
index ecf691428e0..ff0143c1c07 100644
--- a/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score_trimmer.clj
@@ -6,9 +6,7 @@
[clojurewerkz.quartzite.triggers :as triggers]
[metabase-enterprise.data-complexity-score.db :as data-complexity-score.db]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -26,9 +24,11 @@
(let [deleted (data-complexity-score.db/delete-scores-older-than! retention-months)]
(log/infof "Data Complexity Score cleanup complete. Deleted %d rows." (or deleted 0))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete old Data Complexity Score snapshots"}
- DataComplexityScoreTrimmer [_ctx]
+(task/defjob DataComplexityScoreTrimmer
+ "Delete old Data Complexity Score snapshots"
+ {:saved-class "metabase_enterprise.data_complexity_score.task.complexity_score_trimmer.DataComplexityScoreTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-complexity-score-data!))
(defmethod task/init! ::DataComplexityScoreTrimmer [_]
diff --git a/enterprise/backend/src/metabase_enterprise/dependencies/task/backfill.clj b/enterprise/backend/src/metabase_enterprise/dependencies/task/backfill.clj
index 38413b7234b..9278b76bdae 100644
--- a/enterprise/backend/src/metabase_enterprise/dependencies/task/backfill.clj
+++ b/enterprise/backend/src/metabase_enterprise/dependencies/task/backfill.clj
@@ -182,10 +182,11 @@
active-triggers-count (try (count (qs/get-triggers-of-job scheduler job-key)) (catch Exception _))]
(log/infof "Executing %s job. %s total triggers active." job-class active-triggers-count)))
-(task/defjob
- ^{:doc "Backfill the dependency table."
- org.quartz.DisallowConcurrentExecution true}
- BackfillDependencies [ctx]
+(task/defjob BackfillDependencies
+ "Backfill the dependency table."
+ {:saved-class "metabase_enterprise.dependencies.task.backfill.BackfillDependencies"
+ :concurrent? false}
+ [ctx]
(let [ctx ^JobExecutionContext ctx]
(log-job-start ctx)
(run-and-reschedule! (.getScheduler ctx))))
diff --git a/enterprise/backend/src/metabase_enterprise/dependencies/task/entity_check.clj b/enterprise/backend/src/metabase_enterprise/dependencies/task/entity_check.clj
index 7cb7a386573..17c5a849ea2 100644
--- a/enterprise/backend/src/metabase_enterprise/dependencies/task/entity_check.clj
+++ b/enterprise/backend/src/metabase_enterprise/dependencies/task/entity_check.clj
@@ -72,10 +72,11 @@
(deps.settings/dependency-entity-check-delay-minutes)
(deps.settings/dependency-entity-check-variance-minutes)))))
-(task/defjob
- ^{:doc "Check all entities for validity"
- org.quartz.DisallowConcurrentExecution true}
- DependencyEntityCheck [ctx]
+(task/defjob DependencyEntityCheck
+ "Check all entities for validity"
+ {:saved-class "metabase_enterprise.dependencies.task.entity_check.DependencyEntityCheck"
+ :concurrent? false}
+ [ctx]
(log/info "Executing DependencyEntityCheck job...")
(check-entities!)
(reschedule-after-run! (.getScheduler ctx)))
diff --git a/enterprise/backend/src/metabase_enterprise/entity_retrieval/task/sync.clj b/enterprise/backend/src/metabase_enterprise/entity_retrieval/task/sync.clj
index 3b401d86adc..707bf29f7c4 100644
--- a/enterprise/backend/src/metabase_enterprise/entity_retrieval/task/sync.clj
+++ b/enterprise/backend/src/metabase_enterprise/entity_retrieval/task/sync.clj
@@ -21,9 +21,11 @@
(set! *warn-on-reflection* true)
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Reconciles the library entity index (pgvector) with the appdb."}
- OsiAiContextSync [_ctx]
+(task/defjob OsiAiContextSync
+ "Reconciles the library entity index (pgvector) with the appdb."
+ {:saved-class "metabase_enterprise.entity_retrieval.task.sync.OsiAiContextSync"
+ :concurrent? false}
+ [_ctx]
(when (entity-retrieval.core/available?)
(try
;; Full reconcile via the shared coalescing schedule: it waits out a concurrent node rather than
diff --git a/enterprise/backend/src/metabase_enterprise/mcp/task/mcp_usage_trimmer.clj b/enterprise/backend/src/metabase_enterprise/mcp/task/mcp_usage_trimmer.clj
index 02d079428bb..d1df29a3a55 100644
--- a/enterprise/backend/src/metabase_enterprise/mcp/task/mcp_usage_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/mcp/task/mcp_usage_trimmer.clj
@@ -11,9 +11,7 @@
[metabase-enterprise.mcp.db :as mcp.db]
[metabase.metabot.settings :as metabot.settings]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -35,9 +33,11 @@
(log/infof "MCP usage log cleanup complete. Deleted %d tool-call and %d session rows."
(or calls 0) (or sessions 0)))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete old MCP usage log rows"}
- McpUsageTrimmer [_ctx]
+(task/defjob McpUsageTrimmer
+ "Delete old MCP usage log rows"
+ {:saved-class "metabase_enterprise.mcp.task.mcp_usage_trimmer.McpUsageTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-mcp-usage-data!))
(defmethod task/init! ::McpUsageTrimmer
diff --git a/enterprise/backend/src/metabase_enterprise/metabot/task/ai_usage_trimmer.clj b/enterprise/backend/src/metabase_enterprise/metabot/task/ai_usage_trimmer.clj
index fed977d6076..7f533acbe50 100644
--- a/enterprise/backend/src/metabase_enterprise/metabot/task/ai_usage_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/metabot/task/ai_usage_trimmer.clj
@@ -8,9 +8,7 @@
[metabase-enterprise.metabot.db :as metabot.db]
[metabase.metabot.settings :as metabot.settings]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -28,9 +26,11 @@
deleted (metabot.db/delete-usage-logs-created-before! cutoff)]
(log/infof "AI usage log cleanup complete. Deleted %d rows." (or deleted 0)))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete old ai_usage_log rows"}
- AiUsageTrimmer [_ctx]
+(task/defjob AiUsageTrimmer
+ "Delete old ai_usage_log rows"
+ {:saved-class "metabase_enterprise.metabot.task.ai_usage_trimmer.AiUsageTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-usage-data!))
(defmethod task/init! ::AiUsageTrimmer
diff --git a/enterprise/backend/src/metabase_enterprise/remote_sync/task/import.clj b/enterprise/backend/src/metabase_enterprise/remote_sync/task/import.clj
index eb20b796f0f..7fd29606ccc 100644
--- a/enterprise/backend/src/metabase_enterprise/remote_sync/task/import.clj
+++ b/enterprise/backend/src/metabase_enterprise/remote_sync/task/import.clj
@@ -54,7 +54,10 @@
(impl/publish-sync-event! :event/remote-sync-import task-id
{:branch branch :auto true} nil)))))))))))
-(task/defjob ^{:doc "Auto-imports any remote collections."} AutoImport [_]
+(task/defjob AutoImport
+ "Auto-imports any remote collections."
+ {:saved-class "metabase_enterprise.remote_sync.task.import.AutoImport"}
+ [_]
(auto-import!))
(def ^:private auto-import-job-key "metabase.task.remote-sync.auto-import.job")
diff --git a/enterprise/backend/src/metabase_enterprise/remote_sync/task/table_cleanup.clj b/enterprise/backend/src/metabase_enterprise/remote_sync/task/table_cleanup.clj
index bbf5207e09e..2e2b1f58499 100644
--- a/enterprise/backend/src/metabase_enterprise/remote_sync/task/table_cleanup.clj
+++ b/enterprise/backend/src/metabase_enterprise/remote_sync/task/table_cleanup.clj
@@ -7,9 +7,7 @@
[java-time.api :as t]
[metabase-enterprise.remote-sync.db :as remote-sync.db]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -38,9 +36,11 @@
[]
(trim-remote-sync-tasks!))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Clean up old records from remote_sync_task table"}
- RemoteSyncTableCleanup [_ctx]
+(task/defjob RemoteSyncTableCleanup
+ "Clean up old records from remote_sync_task table"
+ {:saved-class "metabase_enterprise.remote_sync.task.table_cleanup.RemoteSyncTableCleanup"
+ :concurrent? false}
+ [_ctx]
(trim-tables!))
(defmethod task/init! ::RemoteSyncTableCleanup
diff --git a/enterprise/backend/src/metabase_enterprise/replacement/timeout.clj b/enterprise/backend/src/metabase_enterprise/replacement/timeout.clj
index 11091251ba6..6801c28d965 100644
--- a/enterprise/backend/src/metabase_enterprise/replacement/timeout.clj
+++ b/enterprise/backend/src/metabase_enterprise/replacement/timeout.clj
@@ -18,9 +18,11 @@
(catch Throwable t
(log/errorf "Error timing out old source replacement runs: %s" (ex-message t)))))
-(task/defjob ^{:doc "Timeout long-running source replacement runs."
- org.quartz.DisallowConcurrentExecution true}
- TimeoutReplacementRuns [ctx]
+(task/defjob TimeoutReplacementRuns
+ "Timeout long-running source replacement runs."
+ {:saved-class "metabase_enterprise.replacement.timeout.TimeoutReplacementRuns"
+ :concurrent? false}
+ [ctx]
(timeout-replacement-runs! ctx))
(defn- start-job! []
diff --git a/enterprise/backend/src/metabase_enterprise/security_center/task/sync_advisories.clj b/enterprise/backend/src/metabase_enterprise/security_center/task/sync_advisories.clj
index 4b308359903..d96c30fe915 100644
--- a/enterprise/backend/src/metabase_enterprise/security_center/task/sync_advisories.clj
+++ b/enterprise/backend/src/metabase_enterprise/security_center/task/sync_advisories.clj
@@ -15,9 +15,7 @@
[metabase-enterprise.security-center.settings :as settings]
[metabase.premium-features.core :as premium-features]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -75,8 +73,11 @@
(log/warnf "Error sending repeat notifications: %s" (ex-message e))))
(metrics/refresh-metrics!)))
-(task/defjob ^{:doc "Periodically fetch and re-evaluate security advisories."
- DisallowConcurrentExecution true} SyncAdvisories [_]
+(task/defjob SyncAdvisories
+ "Periodically fetch and re-evaluate security advisories."
+ {:saved-class "metabase_enterprise.security_center.task.sync_advisories.SyncAdvisories"
+ :concurrent? false}
+ [_]
(sync-and-evaluate!))
(defmethod task/init! ::SyncAdvisories [_]
diff --git a/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_cleanup.clj b/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_cleanup.clj
index 2efd42bd00f..36723c331a4 100644
--- a/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_cleanup.clj
+++ b/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_cleanup.clj
@@ -16,9 +16,7 @@
[metabase.task.core :as task]
[metabase.util.log :as log]
[next.jdbc :as jdbc]
- [next.jdbc.result-set :as jdbc.rs])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [next.jdbc.result-set :as jdbc.rs]))
(set! *warn-on-reflection* true)
@@ -258,9 +256,11 @@
(def ^:private cleanup-job-key (jobs/key "metabase.task.semantic-index-cleanup.job"))
(def ^:private cleanup-trigger-key (triggers/key "metabase.task.semantic-index-cleanup.trigger"))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Clean up inactive semantic search index tables"}
- SemanticIndexCleanup [_ctx]
+(task/defjob SemanticIndexCleanup
+ "Clean up inactive semantic search index tables"
+ {:saved-class "metabase_enterprise.semantic_search.task.index_cleanup.SemanticIndexCleanup"
+ :concurrent? false}
+ [_ctx]
(cleanup-stale-indexes-and-gate-tombstones!))
(defmethod task/init! ::SemanticIndexCleanup [_]
diff --git a/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_repair.clj b/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_repair.clj
index eef162fb4e3..9fe5d5e03f8 100644
--- a/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_repair.clj
+++ b/enterprise/backend/src/metabase_enterprise/semantic_search/task/index_repair.clj
@@ -10,9 +10,7 @@
[metabase.lib-be.core :as lib-be]
[metabase.search.ingestion :as search.ingestion]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -37,9 +35,11 @@
(semantic.health/report-repair-metrics! nil)
(log/errorf "Failed to complete semantic search index repair: %s" (ex-message e)))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Runs repair-index! to maintain semantic search consistency"}
- SemanticIndexRepair [_ctx]
+(task/defjob SemanticIndexRepair
+ "Runs repair-index! to maintain semantic search consistency"
+ {:saved-class "metabase_enterprise.semantic_search.task.index_repair.SemanticIndexRepair"
+ :concurrent? false}
+ [_ctx]
(when (semantic.u/semantic-search-active?)
(log/with-context {:quartz-job-type 'SemanticIndexRepair}
(repair-index!))))
diff --git a/enterprise/backend/src/metabase_enterprise/semantic_search/task/indexer.clj b/enterprise/backend/src/metabase_enterprise/semantic_search/task/indexer.clj
index af3b246bceb..9beba1ccf9c 100644
--- a/enterprise/backend/src/metabase_enterprise/semantic_search/task/indexer.clj
+++ b/enterprise/backend/src/metabase_enterprise/semantic_search/task/indexer.clj
@@ -15,8 +15,7 @@
[metabase.task.core :as task]
[metabase.util.log :as log])
(:import (java.time Duration Instant)
- (java.util Date)
- (org.quartz DisallowConcurrentExecution)))
+ (java.util Date)))
(set! *warn-on-reflection* true)
@@ -35,9 +34,10 @@
(contains? search.config/hnsw-index-backed-strategies
(semantic.settings/semantic-search-vector-strategy)))
-(deftype ^{DisallowConcurrentExecution true
- :doc "Runs an indexer process for a time, expects to be rescheduled to continue"}
- SemanticSearchIndexer []
+(task/defjob-type SemanticSearchIndexer
+ "Runs an indexer process for a time, expects to be rescheduled to continue"
+ {:saved-class "metabase_enterprise.semantic_search.task.indexer.SemanticSearchIndexer"
+ :concurrent? false}
org.quartz.Job
(execute [_ _]
(when (semantic.u/semantic-search-active?)
diff --git a/enterprise/backend/src/metabase_enterprise/semantic_search/task/metric_collector.clj b/enterprise/backend/src/metabase_enterprise/semantic_search/task/metric_collector.clj
index 0b8f376e68e..32117881694 100644
--- a/enterprise/backend/src/metabase_enterprise/semantic_search/task/metric_collector.clj
+++ b/enterprise/backend/src/metabase_enterprise/semantic_search/task/metric_collector.clj
@@ -16,8 +16,7 @@
[next.jdbc.result-set :as jdbc.rs])
(:import
(java.time Instant)
- (java.util Date)
- (org.quartz DisallowConcurrentExecution)))
+ (java.util Date)))
(set! *warn-on-reflection* true)
@@ -83,9 +82,11 @@
;; Keep this outside the try: ordinary failures continue here, while interruption and fatal errors exit.
(search.index-health/refresh-search-index-metrics!))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Collect expensive semantic search metrics"}
- SemanticMetricCollector [_ctx]
+(task/defjob SemanticMetricCollector
+ "Collect expensive semantic search metrics"
+ {:saved-class "metabase_enterprise.semantic_search.task.metric_collector.SemanticMetricCollector"
+ :concurrent? false}
+ [_ctx]
(collect-metrics!))
(def ^:private job-interval-ms (* 10 60 1000))
diff --git a/enterprise/backend/src/metabase_enterprise/semantic_search/task/usage_trimmer.clj b/enterprise/backend/src/metabase_enterprise/semantic_search/task/usage_trimmer.clj
index b86688c1529..fa64d609629 100644
--- a/enterprise/backend/src/metabase_enterprise/semantic_search/task/usage_trimmer.clj
+++ b/enterprise/backend/src/metabase_enterprise/semantic_search/task/usage_trimmer.clj
@@ -8,8 +8,7 @@
[metabase.task.core :as task]
[metabase.util.log :as log])
(:import
- (java.sql Timestamp)
- (org.quartz DisallowConcurrentExecution)))
+ (java.sql Timestamp)))
(set! *warn-on-reflection* true)
@@ -26,9 +25,11 @@
(semantic-search.db/delete-token-tracking-created-before! t))
(log/info "Semantic search old data cleanup successful.")))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Clean up inactive semantic search index tables"}
- SemanticSearchUsageTrimmer [_ctx]
+(task/defjob SemanticSearchUsageTrimmer
+ "Clean up inactive semantic search index tables"
+ {:saved-class "metabase_enterprise.semantic_search.task.usage_trimmer.SemanticSearchUsageTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-token-data!))
(defmethod task/init! ::SemanticSearchUsageTrimmer
diff --git a/enterprise/backend/src/metabase_enterprise/sso/task/delete_expired_relay_state.clj b/enterprise/backend/src/metabase_enterprise/sso/task/delete_expired_relay_state.clj
index 4e3ef7d4bc6..ec733ea1097 100644
--- a/enterprise/backend/src/metabase_enterprise/sso/task/delete_expired_relay_state.clj
+++ b/enterprise/backend/src/metabase_enterprise/sso/task/delete_expired_relay_state.clj
@@ -12,9 +12,11 @@
(set! *warn-on-reflection* true)
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Delete expired SAML SSO RelayState entries"}
- DeleteExpiredSsoRelayState [_ctx]
+(task/defjob DeleteExpiredSsoRelayState
+ "Delete expired SAML SSO RelayState entries"
+ {:saved-class "metabase_enterprise.sso.task.delete_expired_relay_state.DeleteExpiredSsoRelayState"
+ :concurrent? false}
+ [_ctx]
(let [deleted (relay-state/delete-expired!)]
(when (pos? deleted)
(log/debugf "Deleted %d expired SSO RelayState entries" deleted))))
diff --git a/enterprise/backend/src/metabase_enterprise/support_access_grants/task/expire_grants.clj b/enterprise/backend/src/metabase_enterprise/support_access_grants/task/expire_grants.clj
index 17fb82f135e..155a6568073 100644
--- a/enterprise/backend/src/metabase_enterprise/support_access_grants/task/expire_grants.clj
+++ b/enterprise/backend/src/metabase_enterprise/support_access_grants/task/expire_grants.clj
@@ -14,9 +14,11 @@
(set! *warn-on-reflection* true)
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Revoke support access once its grant has ended"}
- ExpireSupportAccessGrants [_ctx]
+(task/defjob ExpireSupportAccessGrants
+ "Revoke support access once its grant has ended"
+ {:saved-class "metabase_enterprise.support_access_grants.task.expire_grants.ExpireSupportAccessGrants"
+ :concurrent? false}
+ [_ctx]
(grants/expire-ended-grants!))
(def ^:private job
diff --git a/src/metabase/analytics/task/send_anonymous_stats.clj b/src/metabase/analytics/task/send_anonymous_stats.clj
index f6de6b65355..69e2fdb1909 100644
--- a/src/metabase/analytics/task/send_anonymous_stats.clj
+++ b/src/metabase/analytics/task/send_anonymous_stats.clj
@@ -11,7 +11,10 @@
(set! *warn-on-reflection* true)
-(task/defjob ^{:doc "If we can collect usage data, do so and send it home"} SendAnonymousUsageStats [_]
+(task/defjob SendAnonymousUsageStats
+ "If we can collect usage data, do so and send it home"
+ {:saved-class "metabase.analytics.task.send_anonymous_stats.SendAnonymousUsageStats"}
+ [_]
(when (analytics.settings/anon-tracking-enabled)
(log/debug "Sending anonymous usage stats.")
(try
diff --git a/src/metabase/audit_app/task/partitions.clj b/src/metabase/audit_app/task/partitions.clj
index 904cb77d8db..54afabd45da 100644
--- a/src/metabase/audit_app/task/partitions.clj
+++ b/src/metabase/audit_app/task/partitions.clj
@@ -101,8 +101,11 @@
;;; scheduling the jobs
-(task/defjob ^:private ^{org.quartz.DisallowConcurrentExecution true}
- ManagePartitions [_]
+(task/defjob ManagePartitions
+ "Creates and drops the partitions of `query_execution`, under Postgres."
+ {:saved-class "metabase.audit_app.task.partitions.ManagePartitions"
+ :concurrent? false}
+ [_]
(when (= :postgres (mdb/db-type)) ; mysql/h2 don't have partitions
(let [retention-days (settings/audit-max-retention-days)]
(with-open [conn (.getConnection (mdb/data-source))]
diff --git a/src/metabase/audit_app/task/truncate_audit_tables.clj b/src/metabase/audit_app/task/truncate_audit_tables.clj
index e351a7ab8e1..4434aecfe01 100644
--- a/src/metabase/audit_app/task/truncate_audit_tables.clj
+++ b/src/metabase/audit_app/task/truncate_audit_tables.clj
@@ -68,7 +68,10 @@
(truncate-table! model timestamp-col)))
(audit-models-to-truncate)))
-(task/defjob ^{:doc "Triggers the removal of `query_execution` rows older than the configured threshold."} TruncateAuditTables [_]
+(task/defjob TruncateAuditTables
+ "Triggers the removal of `query_execution` rows older than the configured threshold."
+ {:saved-class "metabase.audit_app.task.truncate_audit_tables.TruncateAuditTables"}
+ [_]
(truncate-audit-tables!))
(def ^:private truncate-audit-tables-job-key "metabase.task.truncate-audit-tables.job")
diff --git a/src/metabase/channel/task/refresh_slack_channel_user_cache.clj b/src/metabase/channel/task/refresh_slack_channel_user_cache.clj
index f2d38a3accf..63bbf782206 100644
--- a/src/metabase/channel/task/refresh_slack_channel_user_cache.clj
+++ b/src/metabase/channel/task/refresh_slack_channel_user_cache.clj
@@ -27,9 +27,16 @@
(def ^:private startup-job-key "metabase.task.on-startup-refresh-channel-cache.job")
(def ^:private startup-trigger-key "metabase.task.on-startup-refresh-channel-cache.trigger")
-(task/defjob ^{:doc "General slack cache refresh job"} RefreshCache [_] (job))
+(task/defjob RefreshCache
+ "General slack cache refresh job"
+ {:saved-class "metabase.channel.task.refresh_slack_channel_user_cache.RefreshCache"}
+ [_]
+ (job))
-(task/defjob ^{:doc "Startup cache refresh, with cleanup on failure."} RefreshCacheOnStartup [_]
+(task/defjob RefreshCacheOnStartup
+ "Startup cache refresh, with cleanup on failure."
+ {:saved-class "metabase.channel.task.refresh_slack_channel_user_cache.RefreshCacheOnStartup"}
+ [_]
(try (job)
(finally
(task/delete-task! (jobs/key startup-job-key)
diff --git a/src/metabase/explorations/task/collect_orphaned_results.clj b/src/metabase/explorations/task/collect_orphaned_results.clj
index 29a13ff3b64..9561c164489 100644
--- a/src/metabase/explorations/task/collect_orphaned_results.clj
+++ b/src/metabase/explorations/task/collect_orphaned_results.clj
@@ -79,8 +79,10 @@
;; than spin on the same batch.
total))))))
-(task/defjob ^{:doc "Collects exploration result blobs that nothing references any more."}
- CollectOrphanedExplorationResults [_]
+(task/defjob CollectOrphanedExplorationResults
+ "Collects exploration result blobs that nothing references any more."
+ {:saved-class "metabase.explorations.task.collect_orphaned_results.CollectOrphanedExplorationResults"}
+ [_]
(task-history/with-task-history {:task "collect-orphaned-exploration-results"}
(collect-orphaned-results!)))
diff --git a/src/metabase/health_inspector/core.clj b/src/metabase/health_inspector/core.clj
index 55ea6e6261c..af8a1119c17 100644
--- a/src/metabase/health_inspector/core.clj
+++ b/src/metabase/health_inspector/core.clj
@@ -134,7 +134,11 @@
[limit]
(health-inspector.db/latest-runs limit))
-(task/defjob ^:private ^{org.quartz.DisallowConcurrentExecution true} SaveReport [_]
+(task/defjob SaveReport
+ "Runs every health check and saves the results, when the health inspector is enabled."
+ {:saved-class "metabase.health_inspector.core.SaveReport"
+ :concurrent? false}
+ [_]
(when (setting/health-inspector-enabled)
;; background job should always be the lowest priority
(.setPriority (Thread/currentThread) 1)
diff --git a/src/metabase/indexed_entities/task/index_values.clj b/src/metabase/indexed_entities/task/index_values.clj
index e681a28e788..9e103b13020 100644
--- a/src/metabase/indexed_entities/task/index_values.clj
+++ b/src/metabase/indexed_entities/task/index_values.clj
@@ -46,9 +46,11 @@
(indexed-entities.db/delete-model-index! model-index-id)))
(model-index/add-values! model-index))))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Refresh model indexed columns"}
- ModelIndexRefresh [job-context]
+(task/defjob ModelIndexRefresh
+ "Refresh model indexed columns"
+ {:saved-class "metabase.indexed_entities.task.index_values.ModelIndexRefresh"
+ :concurrent? false}
+ [job-context]
(let [{:strs [model-index-id]} (qc/from-job-data job-context)]
(refresh-index! model-index-id)))
diff --git a/src/metabase/mcp/task/mcp_query_handle_gc.clj b/src/metabase/mcp/task/mcp_query_handle_gc.clj
index 38a08b98342..e354cb992f8 100644
--- a/src/metabase/mcp/task/mcp_query_handle_gc.clj
+++ b/src/metabase/mcp/task/mcp_query_handle_gc.clj
@@ -14,9 +14,7 @@
[metabase.mcp.models.mcp-query-handle]
[metabase.mcp.settings :as mcp.settings]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -31,9 +29,11 @@
(log/infof "MCP query handle GC complete. Deleted %d handle(s) older than %d hours."
(or deleted 0) (long ttl-hours))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete expired MCP query handles"}
- McpQueryHandleGc [_ctx]
+(task/defjob McpQueryHandleGc
+ "Delete expired MCP query handles"
+ {:saved-class "metabase.mcp.task.mcp_query_handle_gc.McpQueryHandleGc"
+ :concurrent? false}
+ [_ctx]
(gc-expired-query-handles!))
(defmethod task/init! ::McpQueryHandleGc
diff --git a/src/metabase/metabot/task/conversation_title_backfill.clj b/src/metabase/metabot/task/conversation_title_backfill.clj
index daae1dfd5c5..1dfa16fe218 100644
--- a/src/metabase/metabot/task/conversation_title_backfill.clj
+++ b/src/metabase/metabot/task/conversation_title_backfill.clj
@@ -19,7 +19,7 @@
(java.time Instant)
(java.util Date)
(java.util.concurrent Future TimeoutException TimeUnit)
- (org.quartz DisallowConcurrentExecution JobExecutionContext)))
+ (org.quartz JobExecutionContext)))
(set! *warn-on-reflection* true)
@@ -149,9 +149,11 @@
:paused (when-not (event-driven-pause? reason) paused-retry-delay-seconds)
nil))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Backfill missing Metabot conversation titles."}
- ConversationTitleBackfill [ctx]
+(task/defjob ConversationTitleBackfill
+ "Backfill missing Metabot conversation titles."
+ {:saved-class "metabase.metabot.task.conversation_title_backfill.ConversationTitleBackfill"
+ :concurrent? false}
+ [ctx]
(let [ctx ^JobExecutionContext ctx
after-id (get (qc/from-job-data ctx) "after-id")
result (run-backfill-page! after-id)]
diff --git a/src/metabase/metabot/task/metabot_conversation_trimmer.clj b/src/metabase/metabot/task/metabot_conversation_trimmer.clj
index 318368a3c09..2749a2a066e 100644
--- a/src/metabase/metabot/task/metabot_conversation_trimmer.clj
+++ b/src/metabase/metabot/task/metabot_conversation_trimmer.clj
@@ -12,9 +12,7 @@
[metabase.metabot.db :as metabot.db]
[metabase.metabot.settings :as metabot.settings]
[metabase.task.core :as task]
- [metabase.util.log :as log])
- (:import
- (org.quartz DisallowConcurrentExecution)))
+ [metabase.util.log :as log]))
(set! *warn-on-reflection* true)
@@ -33,9 +31,11 @@
(log/infof "Metabot conversation cleanup complete. Deleted %d conversations (messages/feedback removed by ON DELETE CASCADE)."
(or deleted 0)))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Delete old metabot_conversation rows (cascades to metabot_message)"}
- MetabotConversationTrimmer [_ctx]
+(task/defjob MetabotConversationTrimmer
+ "Delete old metabot_conversation rows (cascades to metabot_message)"
+ {:saved-class "metabase.metabot.task.metabot_conversation_trimmer.MetabotConversationTrimmer"
+ :concurrent? false}
+ [_ctx]
(trim-old-conversations!))
(defmethod task/init! ::MetabotConversationTrimmer
diff --git a/src/metabase/metabot/task/suggested_prompts_generator.clj b/src/metabase/metabot/task/suggested_prompts_generator.clj
index ded768536ae..5d50f5165c1 100644
--- a/src/metabase/metabot/task/suggested_prompts_generator.clj
+++ b/src/metabase/metabot/task/suggested_prompts_generator.clj
@@ -13,8 +13,7 @@
[metabase.util.log :as log])
(:import
(java.time Instant)
- (java.util Date)
- (org.quartz DisallowConcurrentExecution)))
+ (java.util Date)))
(set! *warn-on-reflection* true)
@@ -66,9 +65,11 @@
(when-not (= :api-key-missing (:error-code (ex-data e)))
(log/errorf "Suggested prompts generation failed: %s" (.getMessage e))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Initial _suggested prompts_ generation for the enabled built-in Metabot instances."}
- SuggestedPromptsGenerator [_ctx]
+(task/defjob SuggestedPromptsGenerator
+ "Initial _suggested prompts_ generation for the enabled built-in Metabot instances."
+ {:saved-class "metabase.metabot.task.suggested_prompts_generator.SuggestedPromptsGenerator"
+ :concurrent? false}
+ [_ctx]
(maybe-generate-suggested-prompts!))
(defmethod task/init! ::SuggestedPromptsGenerator
diff --git a/src/metabase/metabot/task/suggested_prompts_refresh.clj b/src/metabase/metabot/task/suggested_prompts_refresh.clj
index f23f11ffa4b..b584d3a9d46 100644
--- a/src/metabase/metabot/task/suggested_prompts_refresh.clj
+++ b/src/metabase/metabot/task/suggested_prompts_refresh.clj
@@ -19,7 +19,7 @@
(:import
(java.time Instant)
(java.util Date)
- (org.quartz DisallowConcurrentExecution ObjectAlreadyExistsException)))
+ (org.quartz ObjectAlreadyExistsException)))
(set! *warn-on-reflection* true)
@@ -56,9 +56,11 @@
(catch Throwable e
(log/warnf "Failed to regenerate suggested prompts for Metabot %s: %s" metabot-id (ex-message e))))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Regenerate a Metabot's suggested prompts after its content scope changes."}
- SuggestedPromptsRefresh [ctx]
+(task/defjob SuggestedPromptsRefresh
+ "Regenerate a Metabot's suggested prompts after its content scope changes."
+ {:saved-class "metabase.metabot.task.suggested_prompts_refresh.SuggestedPromptsRefresh"
+ :concurrent? false}
+ [ctx]
(when-let [metabot-id (get (qc/from-job-data ctx) "metabot-id")]
(regenerate! metabot-id)))
diff --git a/src/metabase/model_persistence/task/persist_refresh.clj b/src/metabase/model_persistence/task/persist_refresh.clj
index ba3245fe166..e211e8b9949 100644
--- a/src/metabase/model_persistence/task/persist_refresh.clj
+++ b/src/metabase/model_persistence/task/persist_refresh.clj
@@ -225,14 +225,18 @@
[_job-context]
(prune-all-deletable! dispatching-refresher))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Refresh persisted tables job"}
- PersistenceRefresh [job-context]
+(task/defjob PersistenceRefresh
+ "Refresh persisted tables job"
+ {:saved-class "metabase.model_persistence.task.persist_refresh.PersistenceRefresh"
+ :concurrent? false}
+ [job-context]
(refresh-job-fn! job-context))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Remove deletable persisted tables"}
- PersistencePrune [job-context]
+(task/defjob PersistencePrune
+ "Remove deletable persisted tables"
+ {:saved-class "metabase.model_persistence.task.persist_refresh.PersistencePrune"
+ :concurrent? false}
+ [job-context]
(prune-job-fn! job-context))
(def ^:private refresh-job-key
diff --git a/src/metabase/mq/queue/quartz.clj b/src/metabase/mq/queue/quartz.clj
index d3b37dee472..e62a05fda91 100644
--- a/src/metabase/mq/queue/quartz.clj
+++ b/src/metabase/mq/queue/quartz.clj
@@ -129,9 +129,9 @@
;;; ------------------------------------ Waking the acquire loop ------------------------------------
-(task/defjob ^{:doc "Never fires. Exists only to give [[wake-scheduler!]] a durable job key to call
- `resumeJob` on."}
- QueueSlotNudgeJob
+(task/defjob QueueSlotNudgeJob
+ "Never fires. Exists only to give [[wake-scheduler!]] a durable job key to call `resumeJob` on."
+ {:saved-class "metabase.mq.queue.quartz.QueueSlotNudgeJob"}
[_ctx])
(def ^:private nudge-job-key (jobs/key "queue-slot-nudge" job-group))
@@ -228,14 +228,16 @@
(backoff-before-refire!)
(throw (doto (JobExecutionException. t) (.setRefireImmediately true)))))))
-(task/defjob ^{:doc "Delivers a queue batch (non-exclusive queues; runs concurrently)."}
- QueueMessageJob
+(task/defjob QueueMessageJob
+ "Delivers a queue batch (non-exclusive queues; runs concurrently)."
+ {:saved-class "metabase.mq.queue.quartz.QueueMessageJob"}
[ctx]
(deliver-batch! ctx))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Delivers a batch for an :exclusive queue — at most one execution per queue cluster-wide."}
- ExclusiveQueueMessageJob
+(task/defjob ExclusiveQueueMessageJob
+ "Delivers a batch for an :exclusive queue — at most one execution per queue cluster-wide."
+ {:saved-class "metabase.mq.queue.quartz.ExclusiveQueueMessageJob"
+ :concurrent? false}
[ctx]
(deliver-batch! ctx))
diff --git a/src/metabase/mq/task/outbox.clj b/src/metabase/mq/task/outbox.clj
index 4b2c7b65124..d22e8d1cf48 100644
--- a/src/metabase/mq/task/outbox.clj
+++ b/src/metabase/mq/task/outbox.clj
@@ -16,9 +16,10 @@
(def ^:private job-key (jobs/key "metabase.mq.task.outbox.job"))
(def ^:private trigger-key (triggers/key "metabase.mq.task.outbox.trigger"))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Republishes transactional-outbox rows orphaned by a crash."}
- FlushMessageOutbox
+(task/defjob FlushMessageOutbox
+ "Republishes transactional-outbox rows orphaned by a crash."
+ {:saved-class "metabase.mq.task.outbox.FlushMessageOutbox"
+ :concurrent? false}
[_]
(let [n (outbox/recover-outbox!)]
(when (pos? n)
diff --git a/src/metabase/mq/task/queue_reaper.clj b/src/metabase/mq/task/queue_reaper.clj
index 0da2c02f0ec..f4192e5ce2e 100644
--- a/src/metabase/mq/task/queue_reaper.clj
+++ b/src/metabase/mq/task/queue_reaper.clj
@@ -81,9 +81,10 @@
(def ^:private job-key (jobs/key "metabase.mq.task.queue-reaper.job"))
(def ^:private trigger-key (triggers/key "metabase.mq.task.queue-reaper.trigger"))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Drops queue messages no node in the cluster has a listener for (older than the max age)."}
- QueueReaper
+(task/defjob QueueReaper
+ "Drops queue messages no node in the cluster has a listener for (older than the max age)."
+ {:saved-class "metabase.mq.task.queue_reaper.QueueReaper"
+ :concurrent? false}
[ctx]
(if (< (node-uptime-ms) startup-grace-ms)
;; too soon after startup — a recovering cluster may still be draining a backlog of old-but-valid
diff --git a/src/metabase/notification/task/send.clj b/src/metabase/notification/task/send.clj
index 1ed0c0e73ea..c3e7cf8cf78 100644
--- a/src/metabase/notification/task/send.clj
+++ b/src/metabase/notification/task/send.clj
@@ -11,7 +11,7 @@
[metabase.tracing.core :as tracing]
[metabase.util.log :as log])
(:import
- (org.quartz DisallowConcurrentExecution JobExecutionContext)))
+ (org.quartz JobExecutionContext)))
(set! *warn-on-reflection* true)
@@ -50,8 +50,9 @@
(log/warnf "Skipping and deleting trigger for subscription %d because the notification is deactivated" subscription-id)
(notification.task.send-trigger/delete-trigger-for-subscription! subscription-id))))))
-(task/defjob ^{:doc "Triggers that send a notification for a subscription."}
- SendNotification
+(task/defjob SendNotification
+ "Triggers that send a notification for a subscription."
+ {:saved-class "metabase.notification.task.send.SendNotification"}
[context]
(let [{:strs [subscription-id]} (qc/from-job-data context)
^JobExecutionContext ctx context
@@ -73,17 +74,16 @@
subscription-id trigger-key scheduled-fire-time fire-time recovering? refire-count scheduler-id)
(send-notification* subscription-id))))
-(task/defjob
- ^{:doc
- "Find all notification subscriptions with cron schedules and create a trigger for each.
- Run once on startup.
+(task/defjob InitNotificationTriggers
+ "Find all notification subscriptions with cron schedules and create a trigger for each.
+ Run once on startup.
- Context: We've migrated alerts from pulse to notifications, see the `v53.2024-12-12T08:05:00` migration.
- This job is needed to create triggers for all existing notification subscriptions after the migration.
- The fact that it runs on every startup is because we have no way to have it run only once.
- Ideally this should be a migration."
- DisallowConcurrentExecution true}
- InitNotificationTriggers
+ Context: We've migrated alerts from pulse to notifications, see the `v53.2024-12-12T08:05:00` migration.
+ This job is needed to create triggers for all existing notification subscriptions after the migration.
+ The fact that it runs on every startup is because we have no way to have it run only once.
+ Ideally this should be a migration."
+ {:saved-class "metabase.notification.task.send.InitNotificationTriggers"
+ :concurrent? false}
[_context]
(log/info "Initializing SendNotification triggers")
(notification.task.send-trigger/init-send-notification-triggers!))
diff --git a/src/metabase/oauth_server/task/cleanup_expired_tokens.clj b/src/metabase/oauth_server/task/cleanup_expired_tokens.clj
index 81e2441b1ee..7ee9de6c284 100644
--- a/src/metabase/oauth_server/task/cleanup_expired_tokens.clj
+++ b/src/metabase/oauth_server/task/cleanup_expired_tokens.clj
@@ -25,8 +25,10 @@
:refresh-tokens-expired (oauth-server.db/delete-refresh-tokens-expired-before! now)
:refresh-tokens-revoked (oauth-server.db/delete-revoked-refresh-tokens!)}))
-(task/defjob ^{:doc "Delete expired and revoked OAuth tokens and authorization codes."}
- CleanupExpiredOAuthTokens [_]
+(task/defjob CleanupExpiredOAuthTokens
+ "Delete expired and revoked OAuth tokens and authorization codes."
+ {:saved-class "metabase.oauth_server.task.cleanup_expired_tokens.CleanupExpiredOAuthTokens"}
+ [_]
(log/debug "Cleaning up expired/revoked OAuth tokens")
(tracing/with-span :tasks "task.oauth-server.cleanup-expired-tokens.delete" {}
(let [counts (cleanup-expired-tokens!)]
diff --git a/src/metabase/premium_features/task/send_metering.clj b/src/metabase/premium_features/task/send_metering.clj
index dc86a68136b..fe53cbec3aa 100644
--- a/src/metabase/premium_features/task/send_metering.clj
+++ b/src/metabase/premium_features/task/send_metering.clj
@@ -24,8 +24,9 @@
(max min-interval-ms (min max-interval-ms v))
default-interval-ms))
-(task/defjob ^{:doc "Send metering events for billing purposes"}
- SendMeteringEvents
+(task/defjob SendMeteringEvents
+ "Send metering events for billing purposes"
+ {:saved-class "metabase.premium_features.task.send_metering.SendMeteringEvents"}
[_]
(log/debug "Running metering events task")
(token-check/send-metering-events!))
diff --git a/src/metabase/product_feedback/task/creator_sentiment_emails.clj b/src/metabase/product_feedback/task/creator_sentiment_emails.clj
index f19ff108e2d..6f97fff6310 100644
--- a/src/metabase/product_feedback/task/creator_sentiment_emails.clj
+++ b/src/metabase/product_feedback/task/creator_sentiment_emails.clj
@@ -80,7 +80,10 @@
(catch Throwable e
(log/errorf "Problem sending creator sentiment email: %s" (ex-message e))))))))
-(task/defjob ^{:doc "Sends out a monthly survey to a portion of the creators."} CreatorSentimentEmail [_]
+(task/defjob CreatorSentimentEmail
+ "Sends out a monthly survey to a portion of the creators."
+ {:saved-class "metabase.product_feedback.task.creator_sentiment_emails.CreatorSentimentEmail"}
+ [_]
(let [current-week (.get (t/local-date) (.weekOfWeekBasedYear (WeekFields/of (Locale/getDefault))))]
(send-creator-sentiment-emails! current-week)))
diff --git a/src/metabase/product_feedback/task/follow_up_emails.clj b/src/metabase/product_feedback/task/follow_up_emails.clj
index ca69064ece1..09386c74a11 100644
--- a/src/metabase/product_feedback/task/follow_up_emails.clj
+++ b/src/metabase/product_feedback/task/follow_up_emails.clj
@@ -40,7 +40,10 @@
^java.time.temporal.Temporal []
(product-feedback.db/earliest-user-date-joined))
-(task/defjob ^{:doc "Sends out a general 2 week email follow up email"} FollowUpEmail [_]
+(task/defjob FollowUpEmail
+ "Sends out a general 2 week email follow up email"
+ {:saved-class "metabase.product_feedback.task.follow_up_emails.FollowUpEmail"}
+ [_]
;; if we've already sent the follow-up email then we are done
(when-not (product-feedback.settings/follow-up-email-sent)
;; figure out when we consider the instance created
diff --git a/src/metabase/pulse/task/email_remove_legacy_pulse.clj b/src/metabase/pulse/task/email_remove_legacy_pulse.clj
index 9071059b8f8..f1ca962d981 100644
--- a/src/metabase/pulse/task/email_remove_legacy_pulse.clj
+++ b/src/metabase/pulse/task/email_remove_legacy_pulse.clj
@@ -32,8 +32,10 @@
:pulses legacy-pulse
:instanceURL (urls/site-url)})})))))
-(task/defjob ^{:doc "Send email to admins and warn about removal of Pulse in 49, This job will only run once."}
- EmailRemoveLegacyPulse [_ctx]
+(task/defjob EmailRemoveLegacyPulse
+ "Send email to admins and warn about removal of Pulse in 49, This job will only run once."
+ {:saved-class "metabase.pulse.task.email_remove_legacy_pulse.EmailRemoveLegacyPulse"}
+ [_ctx]
(email-remove-legacy-pulse))
(defmethod task/init! ::SendWarnPulseRemovalEmail [_job-name]
diff --git a/src/metabase/pulse/task/send_pulses.clj b/src/metabase/pulse/task/send_pulses.clj
index 96384fcaf33..daa19703f12 100644
--- a/src/metabase/pulse/task/send_pulses.clj
+++ b/src/metabase/pulse/task/send_pulses.clj
@@ -19,7 +19,7 @@
[metabase.tracing.core :as tracing]
[metabase.util.log :as log])
(:import
- (org.quartz DisallowConcurrentExecution JobExecutionContext)))
+ (org.quartz JobExecutionContext)))
(set! *warn-on-reflection* true)
@@ -81,8 +81,9 @@
(send-pulse! pulse-id to-send-enabled-channel-ids)
(log/infof "Skip sending pulse %d because all channels have no recipients" pulse-id))))
-(task/defjob ^{:doc "Triggers that send a pulse to a list of channels at a specific time"}
- SendPulse
+(task/defjob SendPulse
+ "Triggers that send a pulse to a list of channels at a specific time"
+ {:saved-class "metabase.pulse.task.send_pulses.SendPulse"}
[context]
(let [{:strs [pulse-id channel-ids]} (qc/from-job-data context)
^JobExecutionContext ctx context
@@ -104,18 +105,17 @@
pulse-id trigger-key scheduled-fire-time fire-time recovering? refire-count scheduler-id)
(send-pulse!* pulse-id channel-ids))))
-(task/defjob
- ^{:doc
- "Find all notification subscriptions with cron schedules and create a trigger for each.
- Run once on startup.
+(task/defjob InitSendPulseTriggers
+ "Find all notification subscriptions with cron schedules and create a trigger for each.
+ Run once on startup.
- Context: Prior to 50, the SendPulse job has a single trigger that sends all pulses, but in #42316
- We've changed it to one trigger per PulseChannel. We need this job so that users migrate from < 50
- have all the triggers initiated properly.
- The fact that it runs on every startup is because we have no way to have it run only once.
- Ideally this should be a migration."
- DisallowConcurrentExecution true}
- InitSendPulseTriggers
+ Context: Prior to 50, the SendPulse job has a single trigger that sends all pulses, but in #42316 We've
+ changed it to one trigger per PulseChannel.
+ We need this job so that users migrate from < 50 have all the triggers initiated properly.
+ The fact that it runs on every startup is because we have no way to have it run only once.
+ Ideally this should be a migration."
+ {:saved-class "metabase.pulse.task.send_pulses.InitSendPulseTriggers"
+ :concurrent? false}
[_context]
(log/info "Initializing SendPulse triggers for dashboard subscriptions")
(task.send-pulses-trigger/init-dashboard-subscription-triggers!))
diff --git a/src/metabase/run_tracking/task.clj b/src/metabase/run_tracking/task.clj
index 62729c2c6eb..20ca36a0b8d 100644
--- a/src/metabase/run_tracking/task.clj
+++ b/src/metabase/run_tracking/task.clj
@@ -18,9 +18,11 @@
;; populated before a trigger can fire locally.
(defonce ^:private reapers (atom {}))
-(task/defjob ^{:doc "Reap orphaned (stale-heartbeat) runs for a registered run model."
- org.quartz.DisallowConcurrentExecution true}
- RunTrackingReaper [ctx]
+(task/defjob RunTrackingReaper
+ "Reap orphaned (stale-heartbeat) runs for a registered run model."
+ {:saved-class "metabase.run_tracking.task.RunTrackingReaper"
+ :concurrent? false}
+ [ctx]
(let [job-key (.. ^JobExecutionContext ctx getJobDetail getKey getName)
{:keys [reap-fn label]} (@reapers job-key)]
(when-let [reaped (and reap-fn (not-empty (reap-fn)))]
diff --git a/src/metabase/search/task/search_index.clj b/src/metabase/search/task/search_index.clj
index 696d5559601..d256e7a4615 100644
--- a/src/metabase/search/task/search_index.clj
+++ b/src/metabase/search/task/search_index.clj
@@ -13,8 +13,7 @@
[metabase.util.queue :as queue])
(:import
(java.time Instant)
- (java.util Date)
- (org.quartz DisallowConcurrentExecution)))
+ (java.util Date)))
(set! *warn-on-reflection* true)
@@ -40,15 +39,19 @@
(cluster-lock/with-cluster-lock cluster-lock-name
(search/init-index! {:force-reset? false, :re-populate? false})))))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Populate a new Search Index"}
- SearchIndexReindex [_ctx]
+(task/defjob SearchIndexReindex
+ "Populate a new Search Index"
+ {:saved-class "metabase.search.task.search_index.SearchIndexReindex"
+ :concurrent? false}
+ [_ctx]
(cluster-lock/with-cluster-lock cluster-lock-name
(search/reindex! {:async? false})))
-(task/defjob ^{DisallowConcurrentExecution true
- :doc "Ensure indexes exist for the active engines"}
- SearchIndexInit [_ctx]
+(task/defjob SearchIndexInit
+ "Ensure indexes exist for the active engines"
+ {:saved-class "metabase.search.task.search_index.SearchIndexInit"
+ :concurrent? false}
+ [_ctx]
(init!))
;; Atom holding a promise that is delivered when the background init thread finishes.
diff --git a/src/metabase/session/task/session_cleanup.clj b/src/metabase/session/task/session_cleanup.clj
index 742e0a37ea0..48de8b117f8 100644
--- a/src/metabase/session/task/session_cleanup.clj
+++ b/src/metabase/session/task/session_cleanup.clj
@@ -34,9 +34,9 @@
(def ^:private session-cleanup-job-key (jobs/key "metabase.task.session-cleanup.job"))
(def ^:private session-cleanup-trigger-key (triggers/key "metabase.task.session-cleanup.trigger"))
-(task/defjob ^{:doc "Job that records the ending of sessions that stopped being live, and deletes those ended
- long ago."}
- SessionCleanup
+(task/defjob SessionCleanup
+ "Job that records the ending of sessions that stopped being live, and deletes those ended long ago."
+ {:saved-class "metabase.session.task.session_cleanup.SessionCleanup"}
[_]
(cleanup-sessions!)
(session/prune-session-activity-cache!))
diff --git a/src/metabase/sync/task/sync_databases.clj b/src/metabase/sync/task/sync_databases.clj
index 000b707cdd9..a1e10b7345a 100644
--- a/src/metabase/sync/task/sync_databases.clj
+++ b/src/metabase/sync/task/sync_databases.clj
@@ -44,8 +44,9 @@
#(satisfies? qc/JobDataMapConversion %)]]
(u/the-id (get (qc/from-job-data job-context) "db-id")))
-;; The DisallowConcurrentExecution on the two defrecords below attaches an annotation to the generated class that will
-;; constrain the job execution to only be one at a time. Other triggers wanting the job to run will misfire.
+;; The `:concurrent? false` option on the two jobs below puts the `DisallowConcurrentExecution` annotation on
+;; the generated class, which constrains the job execution to only be one at a time. Other triggers wanting
+;; the job to run will misfire.
(def ^:private analyze-duration-threshold-for-refingerprinting
"If the `analyze-db!` step is shorter than this number of `minutes`, then we may refingerprint fields."
@@ -122,9 +123,11 @@
:else
(sync-and-analyze-database*! database-id))))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Sync and analyze the database"}
- SyncAndAnalyzeDatabase [job-context]
+(task/defjob SyncAndAnalyzeDatabase
+ "Sync and analyze the database"
+ {:saved-class "metabase.sync.task.sync_databases.SyncAndAnalyzeDatabase"
+ :concurrent? false}
+ [job-context]
(sync-and-analyze-database! job-context))
(defn- update-field-values!
@@ -143,9 +146,11 @@
(sync.field-values/update-field-values! database)
(log/infof "Skipping update, automatic Field value updates are disabled for Database %d." database-id)))))))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Update field values"}
- UpdateFieldValues [job-context]
+(task/defjob UpdateFieldValues
+ "Update field values"
+ {:saved-class "metabase.sync.task.sync_databases.UpdateFieldValues"
+ :concurrent? false}
+ [job-context]
(update-field-values! job-context))
;;; +----------------------------------------------------------------------------------------------------------------+
diff --git a/src/metabase/task/QUARTZ.md b/src/metabase/task/QUARTZ.md
index dc7120485ae..1bf9a8c44b0 100644
--- a/src/metabase/task/QUARTZ.md
+++ b/src/metabase/task/QUARTZ.md
@@ -13,6 +13,24 @@ In Quartz, there are two fundamental concepts:
Jobs and triggers are created separately and then associated with each other when scheduled in the Quartz system.
+## Defining a job
+
+Define a job with `task/defjob`. It takes a type name, an optional docstring, an options map, an argument vector, and the job's code:
+
+```clojure
+(task/defjob MyJob
+ "What the job does."
+ {:saved-class "metabase.my_module.task.my_job.MyJob"}
+ [job-context]
+ (do-the-work!))
+```
+
+Every job needs a `:saved-class`. It's the exact name of the job's class, which Quartz stores in the app DB and uses to find the job's code. For a new job, leave it out and compile: the error prints the entry to paste, with the name the job's namespace gives it.
+
+Once a job has shipped, keep its `:saved-class` the same, even when you move its namespace or rename its type. During a rolling upgrade, old and new nodes share the Quartz tables, and a node that can't load a stored class name deletes the job and its triggers at startup.
+
+For a job that implements more than `org.quartz.Job`, such as an `InterruptableJob`, use `task/defjob-type`.
+
## Use cases at Metabase
This list is not complete and is open to additions. If you have other use cases or patterns for Quartz tasks in Metabase, please feel free to add them here.
@@ -36,7 +54,10 @@ These tasks execute once when Metabase starts up. They're useful for initializat
(def startup-job-key (jobs/key "my.startup.task.job"))
(def startup-trigger-key (triggers/key "my.startup.task.trigger"))
-(task/defjob StartupTask [_]
+(task/defjob StartupTask
+ "Runs the startup initialization."
+ {:saved-class "my.startup.task.StartupTask"}
+ [_]
(println "Running startup initialization task"))
(defmethod task/init! ::StartupTask [_]
@@ -59,7 +80,10 @@ These tasks run on a recurring schedule defined by a cron expression.
(def daily-job-key (jobs/key "my.scheduled.daily.task.job"))
(def daily-trigger-key (triggers/key "my.scheduled.daily.task.trigger"))
-(task/defjob DailyTask [_]
+(task/defjob DailyTask
+ "Runs the daily scheduled task."
+ {:saved-class "my.scheduled.task.DailyTask"}
+ [_]
(println "Running daily scheduled task"))
(defmethod task/init! ::DailyTask [_]
@@ -105,7 +129,9 @@ You can pass data to jobs using the job data map:
Inside the job, you can access this data:
```clojure
-(task/defjob MyJob [job-context]
+(task/defjob MyJob
+ {:saved-class "my.task.MyJob"}
+ [job-context]
(let [data-map (qc/from-job-data job-context)
db-id (get data-map "db-id")]
;; Use the data...
@@ -126,14 +152,17 @@ Choose the appropriate misfire handling based on your task's requirements.
### Preventing Concurrent Execution
-By default, Quartz allows multiple instances of the same job to run concurrently. To prevent concurrent execution of a job, use the `DisallowConcurrentExecution` annotation:
+By default, Quartz allows multiple instances of the same job to run concurrently. To prevent concurrent execution of a job, give it the `:concurrent? false` option, which puts the Quartz `DisallowConcurrentExecution` annotation on its class:
```clojure
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true} MyNonConcurrentJob [job-context]
+(task/defjob MyNonConcurrentJob
+ {:saved-class "my.task.MyNonConcurrentJob"
+ :concurrent? false}
+ [job-context]
(println "This job will not run concurrently with itself"))
```
-When this annotation is applied, Quartz will block a new instance of the job from starting if a previous instance is still running.
+With this option, Quartz will block a new instance of the job from starting if a previous instance is still running.
This is particularly important for long-running tasks that:
- we need to make sure they don't run concurrently across instances
- operate on shared resources
@@ -143,7 +172,9 @@ This is particularly important for long-running tasks that:
Metabase provides a `rerun-on-error` macro to automatically retry a job if it fails with an exception:
```clojure
-(task/defjob MyJob [job-context]
+(task/defjob MyJob
+ {:saved-class "my.task.MyJob"}
+ [job-context]
(task/rerun-on-error job-context
;; Your job code here
(do-something-that-might-fail)))
diff --git a/src/metabase/task/core.clj b/src/metabase/task/core.clj
index 71449dc0c41..630bd69525d 100644
--- a/src/metabase/task/core.clj
+++ b/src/metabase/task/core.clj
@@ -12,6 +12,7 @@
add-trigger!
add-trigger-listener!
defjob
+ defjob-type
delete-all-triggers-of-job!
delete-task!
delete-trigger!
diff --git a/src/metabase/task/impl.clj b/src/metabase/task/impl.clj
index 54df0b73a58..d2fbdddf891 100644
--- a/src/metabase/task/impl.clj
+++ b/src/metabase/task/impl.clj
@@ -19,7 +19,6 @@
Find the JavaDoc for Quartz here: http://www.quartz-scheduler.org/api/2.3.0/index.html"
(:require
[clojure.string :as str]
- [clojurewerkz.quartzite.jobs :as jobs]
[clojurewerkz.quartzite.scheduler :as qs]
[environ.core :as env]
[metabase.app-db.core :as mdb]
@@ -358,15 +357,175 @@
(log/error msg# (ex-message e#))
(throw (JobExecutionException. msg# e# true))))))
-;; this is the sanctioned metabase.task/defjob wrapper; it must expand to the quartzite macro
-#_{:clj-kondo/ignore [:discouraged-var]}
+(def ^:private job-option-annotations
+ "The Quartz annotation that each boolean job option puts on the job class, and the value that asks for it."
+ {:concurrent? {:annotation 'org.quartz.DisallowConcurrentExecution, :when false}
+ :persist-data? {:annotation 'org.quartz.PersistJobDataAfterExecution, :when true}})
+
+(def ^:private job-option-keys
+ (into #{:saved-class} (keys job-option-annotations)))
+
+(defn- job-definition-error [macro-name type-name message data]
+ (ex-info (format "Invalid (task/%s %s ...): %s" macro-name type-name message)
+ (assoc data :type-name type-name)))
+
+(defn- check-job-options!
+ "Throws unless `options` is a valid options map for the job type `type-name`."
+ [macro-name type-name options]
+ (let [error (partial job-definition-error macro-name type-name)
+ unknown-keys (when (map? options)
+ (sort (remove job-option-keys (keys options))))
+ saved-class (:saved-class options)
+ default-name (str (munge (ns-name *ns*)) "." type-name)]
+ (cond
+ (not (map? options))
+ (throw (error (format (str "its options map is missing. It goes after the optional docstring, and needs"
+ " the class name that Quartz stores for the job. For a new job, add:\n\n"
+ " {:saved-class %s}\n")
+ (pr-str default-name))
+ {}))
+
+ (seq unknown-keys)
+ (throw (error (format "unknown options %s. The options are %s."
+ (pr-str (vec unknown-keys)) (pr-str (vec (sort job-option-keys))))
+ {:unknown-keys unknown-keys}))
+
+ (not (contains? options :saved-class))
+ (throw (error (format (str "it needs the class name that Quartz stores for the job. For a new job, add"
+ " this to its options map:\n\n"
+ " :saved-class %s\n\n"
+ "For a job that moved or was renamed, put back the `:saved-class` it had.")
+ (pr-str default-name))
+ {}))
+
+ (not (string? saved-class))
+ (throw (error "its `:saved-class` must be a string literal." {:saved-class saved-class}))
+
+ (not (re-matches #"[A-Za-z_$][A-Za-z0-9_$]*(\.[A-Za-z_$][A-Za-z0-9_$]*)+" saved-class))
+ (throw (error (format (str "its `:saved-class` %s is not a fully qualified Java class name, such as %s."
+ " A namespace's `-` is a `_` in a class name.")
+ (pr-str saved-class) (pr-str default-name))
+ {:saved-class saved-class}))
+
+ :else
+ (doseq [k (keys job-option-annotations)
+ :let [v (get options k)]
+ :when (and (contains? options k) (not (boolean? v)))]
+ (throw (error (format "its `%s` must be `true` or `false`." k) {k v}))))))
+
+(defn- parse-job-definition
+ "Splits the arguments of [[defjob]] or [[defjob-type]] after the type name into the `:docstring`, the valid
+ `:options` and the `:more` that follows them."
+ [macro-name type-name args]
+ (let [[docstring args] (if (string? (first args))
+ [(first args) (rest args)]
+ [nil args])
+ [options & more] args]
+ (when-not (simple-symbol? type-name)
+ (throw (job-definition-error macro-name type-name
+ "its type name must be a symbol with no namespace."
+ {})))
+ ;; This refuses only the metadata that would be lost, where an annotation is a symbol key.
+ ;; Other metadata is fine: some readers, such as Eastwood's, put the line and column on every symbol.
+ (when (some #(or (symbol? %) (#{:doc :private} %)) (keys (meta type-name)))
+ (throw (job-definition-error macro-name type-name
+ (str "its type name has metadata. Put the description in the docstring,"
+ " and ask for Quartz annotations with the `:concurrent?` and"
+ " `:persist-data?` options.")
+ {:metadata (meta type-name)})))
+ (check-job-options! macro-name type-name options)
+ {:docstring docstring, :options options, :more more}))
+
+(defn- job-class-symbol
+ "The class name in `options` as a symbol, with the annotations that the options ask for as its metadata."
+ [options]
+ (with-meta (symbol (:saved-class options))
+ (into {}
+ (for [[k {:keys [annotation], wanted :when}] job-option-annotations
+ :when (and (contains? options k) (= wanted (get options k)))]
+ [annotation true]))))
+
+(defn- job-type-form
+ "The form that defines the job class for [[defjob]] and [[defjob-type]]."
+ [type-name {:keys [docstring options]} specs]
+ (let [class-symbol (job-class-symbol options)]
+ ;; `deftype` names its class after the current namespace and then expands to `deftype*`, so this skips it.
+ ;; The forms of a top-level `do` are compiled one at a time, so the code that follows can use `type-name`.
+ `(do
+ (deftype* ~(symbol (name (ns-name *ns*)) (name type-name)) ~class-symbol []
+ :implements [~@(filter symbol? specs) clojure.lang.IType]
+ ~@(filter seq? specs))
+ (.importClass ^clojure.lang.Namespace *ns*
+ '~type-name
+ (clojure.lang.RT/classForNameNonLoading ~(:saved-class options)))
+ ;; the factory function that `deftype` and `defrecord` define
+ (defn ~(symbol (str "->" type-name))
+ ~(or docstring (str "Returns a new `" type-name "` job."))
+ []
+ (new ~(with-meta class-symbol nil))))))
+
+(defmacro defjob-type
+ "Defines a Quartz job class from interfaces and method implementations, as `deftype` with no fields does.
+ Prefer [[defjob]], and use this for a job that implements more than `org.quartz.Job`.
+ Nothing is wrapped around the methods: add a log context or a tracing span yourself.
+
+ (task/defjob-type MyJob
+ \"What the job does.\"
+ {:saved-class \"metabase.my_module.task.my_job.MyJob\"}
+ org.quartz.Job
+ (execute [_ ctx] ...)
+ org.quartz.InterruptableJob
+ (interrupt [_] ...))
+
+ The docstring is optional, and the options are those of [[defjob]], with `:saved-class` required."
+ {:arglists '([type-name docstring? options & specs])}
+ [type-name & args]
+ (let [{:keys [more], :as definition} (parse-job-definition "defjob-type" type-name args)]
+ (job-type-form type-name definition more)))
+
+(defn- job-body-form
+ "The `body` of a [[defjob]], inside its log context and its tracing span."
+ [type-name body]
+ `(log/with-context {:quartz-job-type (quote ~type-name)}
+ (tracing/with-span :tasks (str "task." (quote ~type-name)) {:task/name (str (quote ~type-name))}
+ ~@body)))
+
(defmacro defjob
- "Like `clojurewerkz.quartzite.task/defjob` but with a log context and an OpenTelemetry tracing span."
- [jtype args & body]
- `(jobs/defjob ~jtype ~args
- (log/with-context {:quartz-job-type (quote ~jtype)}
- (tracing/with-span :tasks (str "task." (quote ~jtype)) {:task/name (str (quote ~jtype))}
- ~@body))))
+ "Defines a Quartz job class that runs `body` with a log context and an OpenTelemetry tracing span around it.
+ Imports the class as `type-name`, and defines the `->type-name` function that returns a new job.
+
+ (task/defjob MyJob
+ \"What the job does.\"
+ {:saved-class \"metabase.my_module.task.my_job.MyJob\"
+ :concurrent? false}
+ [job-context]
+ ...)
+
+ The docstring is optional. `args` is a vector of one binding, for the `JobExecutionContext`. Options:
+
+ - `:saved-class` (required, a string literal): the exact name of the job's class.
+ Quartz stores this name in the app DB, and finds the job's class by it.
+ Keep it the same when the namespace or `type-name` changes, or stored jobs lose their class.
+ A new job takes the name its namespace would give it, which the error for a missing `:saved-class` prints.
+ A stored job whose class name no longer exists is deleted at startup, with its triggers.
+ - `:concurrent?` (default `true`): `false` stops Quartz running two executions of the job at once, as
+ `org.quartz.DisallowConcurrentExecution` does.
+ - `:persist-data?` (default `false`): `true` saves the job's data map again after each execution, as
+ `org.quartz.PersistJobDataAfterExecution` does.
+
+ For a job that implements more than `org.quartz.Job`, see [[defjob-type]]."
+ {:arglists '([type-name docstring? options args & body])}
+ [type-name & args]
+ (let [{[job-args & body] :more, :as definition} (parse-job-definition "defjob" type-name args)]
+ (when-not (and (vector? job-args) (= 1 (count job-args)))
+ (throw (job-definition-error "defjob" type-name
+ "its argument vector takes one binding, for the `JobExecutionContext`."
+ {:args job-args})))
+ (job-type-form type-name
+ definition
+ `[org.quartz.Job
+ (~'execute [~'_this ~@job-args]
+ ~(job-body-form type-name body))])))
(defn add-job-listener!
"Add a [Quartz Joblistener](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/tutorial-lesson-07.html). That will
diff --git a/src/metabase/task_history/task/task_history_cleanup.clj b/src/metabase/task_history/task/task_history_cleanup.clj
index 1a8d68714e3..a68bfe13ac7 100644
--- a/src/metabase/task_history/task/task_history_cleanup.clj
+++ b/src/metabase/task_history/task/task_history_cleanup.clj
@@ -26,9 +26,10 @@
"Task history cleanup successful, rows were deleted"
"Task history cleanup successful, no rows were deleted"))))))
-(task/defjob
- ^{:doc "Delete older TaskHistory rows -- see docstring of `task-history/cleanup-task-history!` for more details."}
- TaskHistoryCleanup [_]
+(task/defjob TaskHistoryCleanup
+ "Delete older TaskHistory rows -- see docstring of `task-history/cleanup-task-history!` for more details."
+ {:saved-class "metabase.task_history.task.task_history_cleanup.TaskHistoryCleanup"}
+ [_]
(task-history-cleanup!))
(def ^:private job-key "metabase.task.task-history-cleanup.job")
diff --git a/src/metabase/transforms/canceling.clj b/src/metabase/transforms/canceling.clj
index 2327c665e70..327b694c4b0 100644
--- a/src/metabase/transforms/canceling.clj
+++ b/src/metabase/transforms/canceling.clj
@@ -150,9 +150,11 @@
;; is at least visible in metrics. Magnitude can be inferred from log volume if needed.
(analytics/inc! :metabase-transforms/cancelation-completed {:outcome "error"}))))
-(task/defjob ^{:doc "Cancel items that haven't been canceled in two minutes"
- org.quartz.DisallowConcurrentExecution true}
- CancelOldTransformRuns [ctx]
+(task/defjob CancelOldTransformRuns
+ "Cancel items that haven't been canceled in two minutes"
+ {:saved-class "metabase.transforms.canceling.CancelOldTransformRuns"
+ :concurrent? false}
+ [ctx]
(cancel-old-transform-runs! ctx))
(defn- start-job! []
diff --git a/src/metabase/transforms/notification.clj b/src/metabase/transforms/notification.clj
index f37d263beda..2d417590ffb 100644
--- a/src/metabase/transforms/notification.clj
+++ b/src/metabase/transforms/notification.clj
@@ -67,9 +67,11 @@
(def ^:private digest-job-key "metabase.transforms.notification.failure-digest-job")
(def ^:private digest-trigger-key "metabase.transforms.notification.failure-digest-trigger")
-(task/defjob ^{:doc "Sends the daily transform failure digest."
- org.quartz.DisallowConcurrentExecution true}
- SendTransformFailureDigest [_ctx]
+(task/defjob SendTransformFailureDigest
+ "Sends the daily transform failure digest."
+ {:saved-class "metabase.transforms.notification.SendTransformFailureDigest"
+ :concurrent? false}
+ [_ctx]
(send-failure-digest!))
(defmethod task/init! ::SendTransformFailureDigest [_]
diff --git a/src/metabase/transforms/schedule.clj b/src/metabase/transforms/schedule.clj
index 206900ba309..7b363eb77eb 100644
--- a/src/metabase/transforms/schedule.clj
+++ b/src/metabase/transforms/schedule.clj
@@ -80,9 +80,10 @@
(log/info "Deleting trigger for transform job with schedule" (:schedule job-id-or-trigger))
(task/delete-trigger! (-> job-id-or-trigger :key triggers/key))))))
-(task/defjob ^{:doc "Run transforms."
- org.quartz.DisallowConcurrentExecution true}
- RunTransforms
+(task/defjob RunTransforms
+ "Run transforms."
+ {:saved-class "metabase.transforms.schedule.RunTransforms"
+ :concurrent? false}
[context]
(let [job-id (-> (conversion/from-job-data context)
(get "job-id"))
diff --git a/src/metabase/transforms/timeout.clj b/src/metabase/transforms/timeout.clj
index 62e2a6f9a96..b411d907820 100644
--- a/src/metabase/transforms/timeout.clj
+++ b/src/metabase/transforms/timeout.clj
@@ -18,9 +18,11 @@
(when-let [timed-out (not-empty (transform-run/timeout-old-runs! (transforms.settings/transform-timeout) :minute))]
(log/infof "Timed out %d transform run(s)." (count timed-out)))))
-(task/defjob ^{:doc "Timeout long-running tasks that have been lost by a worker."
- org.quartz.DisallowConcurrentExecution true}
- TimeoutTransforms [ctx]
+(task/defjob TimeoutTransforms
+ "Timeout long-running tasks that have been lost by a worker."
+ {:saved-class "metabase.transforms.timeout.TimeoutTransforms"
+ :concurrent? false}
+ [ctx]
(timeout-transform-runs! ctx))
(defn- start-job! []
diff --git a/src/metabase/usage_metadata/task/process.clj b/src/metabase/usage_metadata/task/process.clj
index fbdc4ec127b..8c8eee350c0 100644
--- a/src/metabase/usage_metadata/task/process.clj
+++ b/src/metabase/usage_metadata/task/process.clj
@@ -26,9 +26,10 @@
(def ^:private trigger-key
(triggers/key "metabase.task.usage-metadata-process.trigger"))
-(task/defjob ^{org.quartz.DisallowConcurrentExecution true
- :doc "Process usage metadata rollups from query execution history."}
- UsageMetadataProcess
+(task/defjob UsageMetadataProcess
+ "Process usage metadata rollups from query execution history."
+ {:saved-class "metabase.usage_metadata.task.process.UsageMetadataProcess"
+ :concurrent? false}
[_]
(when (usage-metadata.settings/usage-metadata-enabled?)
(try
diff --git a/src/metabase/version/task/upgrade_checks.clj b/src/metabase/version/task/upgrade_checks.clj
index d24d1932afa..4cbbb2e744e 100644
--- a/src/metabase/version/task/upgrade_checks.clj
+++ b/src/metabase/version/task/upgrade_checks.clj
@@ -30,7 +30,10 @@
(throw (Exception. (format "[%d]: %s" status body))))
(json/decode+kw body)))
-(task/defjob ^{:doc "Simple job which looks up all databases and runs a sync on them"} CheckForNewVersions [_]
+(task/defjob CheckForNewVersions
+ "Simple job which looks up all databases and runs a sync on them"
+ {:saved-class "metabase.version.task.upgrade_checks.CheckForNewVersions"}
+ [_]
(when (version.settings/check-for-updates)
(log/debug "Checking for new Metabase version info.")
(try
diff --git a/test/metabase/app_db/custom_migrations_test.clj b/test/metabase/app_db/custom_migrations_test.clj
index e4d8f979e42..9ed241161ca 100644
--- a/test/metabase/app_db/custom_migrations_test.clj
+++ b/test/metabase/app_db/custom_migrations_test.clj
@@ -82,7 +82,10 @@
(is (= (sort versions)
versions)))))
-(task/defjob AbandonmentEmail [_] :default)
+(task/defjob AbandonmentEmail
+ {:saved-class "metabase.app_db.custom_migrations_test.AbandonmentEmail"}
+ [_]
+ :default)
(defn- table-default [table]
(letfn [(with-timestamped [props]
diff --git a/test/metabase/app_db/quartz_test.clj b/test/metabase/app_db/quartz_test.clj
index e129db54bfa..7f0c0a618c4 100644
--- a/test/metabase/app_db/quartz_test.clj
+++ b/test/metabase/app_db/quartz_test.clj
@@ -1,10 +1,16 @@
(ns metabase.app-db.quartz-test
(:require
+ [clojure.java.io :as io]
+ [clojure.set :as set]
+ [clojure.string :as str]
[clojure.test :refer :all]
[metabase.app-db.connection :as mdb.connection]
[metabase.app-db.connection-pool-setup :as mdb.connection-pool-setup]
[metabase.app-db.data-source :as mdb.data-source]
[metabase.app-db.quartz :as mdb.quartz]
+ [metabase.classloader.core :as classloader]
+ [metabase.config.core :as config]
+ [rewrite-clj.zip :as z]
[toucan2.connection :as t2.conn])
(:import
(com.mchange.v2.c3p0 DataSources)
@@ -68,3 +74,179 @@
(finally
(DataSources/destroy ^javax.sql.DataSource (:data-source app-db))
(DataSources/destroy ^javax.sql.DataSource (:quartz-data-source app-db))))))))))
+
+(defn- job-source-files
+ "Source files of this edition that mention Quartz, which include every file that defines a job class."
+ []
+ (for [dir ["src" "enterprise/backend/src"]
+ :when (or config/ee-available? (= dir "src"))
+ ^java.io.File file (file-seq (io/file dir))
+ :when (and (str/ends-with? (.getName file) ".clj")
+ (re-find #"defjob|org\.quartz|quartzite" (slurp file)))]
+ {:file file
+ :namespace (-> (subs (.getPath file) (inc (count dir)))
+ (str/replace #"\.clj$" "")
+ (str/replace "/" ".")
+ (str/replace "_" "-")
+ symbol)}))
+
+(defn- job-definition-options
+ "Returns the options map of the form at `zloc` if it is a `task/defjob` or a `task/defjob-type`."
+ [zloc]
+ (when (z/list? zloc)
+ (let [head (z/down zloc)]
+ (when (and (= :token (z/tag head))
+ (symbol? (z/sexpr head))
+ (#{"defjob" "defjob-type"} (name (z/sexpr head))))
+ (some-> (z/find head z/right #(= :map (z/tag %)))
+ z/sexpr)))))
+
+(def ^:private job-definitions
+ "The job definitions of this edition, each as its `:namespace`, `:type-name` and `:saved-class`."
+ ;; These come from the source text, so they don't depend on which namespaces are loaded, or were reloaded.
+ (delay
+ (vec (for [{:keys [^java.io.File file], ns-symb :namespace} (job-source-files)
+ :when (str/includes? (slurp file) "defjob")
+ zloc (take-while (complement z/end?) (iterate z/next (z/of-file file)))
+ :let [options (job-definition-options zloc)]
+ :when options]
+ {:namespace ns-symb
+ :type-name (-> zloc z/down z/right z/sexpr)
+ :saved-class (:saved-class options)}))))
+
+(defn- in-this-edition?
+ "Whether this edition has the class named `class-name`."
+ [class-name]
+ (or config/ee-available?
+ ;; a job that moved out of enterprise can keep an enterprise class name
+ (some #(= class-name (:saved-class %)) @job-definitions)
+ (not (str/starts-with? class-name "metabase_enterprise."))))
+
+(def ^:private listed-job-classes
+ "The class name of every job, apart from its `:saved-class`."
+ ;; This is what catches a bulk rename that rewrites a `:saved-class`, which nothing else would notice.
+ #{"metabase.analytics.task.send_anonymous_stats.SendAnonymousUsageStats"
+ "metabase.audit_app.task.partitions.ManagePartitions"
+ "metabase.audit_app.task.truncate_audit_tables.TruncateAuditTables"
+ "metabase.channel.task.refresh_slack_channel_user_cache.RefreshCache"
+ "metabase.channel.task.refresh_slack_channel_user_cache.RefreshCacheOnStartup"
+ "metabase.explorations.task.collect_orphaned_results.CollectOrphanedExplorationResults"
+ "metabase.health_inspector.core.SaveReport"
+ "metabase.indexed_entities.task.index_values.ModelIndexRefresh"
+ "metabase.mcp.task.mcp_query_handle_gc.McpQueryHandleGc"
+ "metabase.metabot.task.conversation_title_backfill.ConversationTitleBackfill"
+ "metabase.metabot.task.metabot_conversation_trimmer.MetabotConversationTrimmer"
+ "metabase.metabot.task.suggested_prompts_generator.SuggestedPromptsGenerator"
+ "metabase.metabot.task.suggested_prompts_refresh.SuggestedPromptsRefresh"
+ "metabase.model_persistence.task.persist_refresh.PersistencePrune"
+ "metabase.model_persistence.task.persist_refresh.PersistenceRefresh"
+ "metabase.mq.queue.quartz.ExclusiveQueueMessageJob"
+ "metabase.mq.queue.quartz.QueueMessageJob"
+ "metabase.mq.queue.quartz.QueueSlotNudgeJob"
+ "metabase.mq.task.outbox.FlushMessageOutbox"
+ "metabase.mq.task.queue_reaper.QueueReaper"
+ "metabase.notification.task.send.InitNotificationTriggers"
+ "metabase.notification.task.send.SendNotification"
+ "metabase.oauth_server.task.cleanup_expired_tokens.CleanupExpiredOAuthTokens"
+ "metabase.premium_features.task.send_metering.SendMeteringEvents"
+ "metabase.product_feedback.task.creator_sentiment_emails.CreatorSentimentEmail"
+ "metabase.product_feedback.task.follow_up_emails.FollowUpEmail"
+ "metabase.pulse.task.email_remove_legacy_pulse.EmailRemoveLegacyPulse"
+ "metabase.pulse.task.send_pulses.InitSendPulseTriggers"
+ "metabase.pulse.task.send_pulses.SendPulse"
+ "metabase.run_tracking.task.RunTrackingReaper"
+ "metabase.search.task.search_index.SearchIndexInit"
+ "metabase.search.task.search_index.SearchIndexReindex"
+ "metabase.session.task.session_cleanup.SessionCleanup"
+ "metabase.sync.task.sync_databases.SyncAndAnalyzeDatabase"
+ "metabase.sync.task.sync_databases.UpdateFieldValues"
+ "metabase.task_history.task.task_history_cleanup.TaskHistoryCleanup"
+ "metabase.transforms.canceling.CancelOldTransformRuns"
+ "metabase.transforms.notification.SendTransformFailureDigest"
+ "metabase.transforms.schedule.RunTransforms"
+ "metabase.transforms.timeout.TimeoutTransforms"
+ "metabase.usage_metadata.task.process.UsageMetadataProcess"
+ "metabase.version.task.upgrade_checks.CheckForNewVersions"
+ "metabase_enterprise.agent_api.task.agent_api_usage_trimmer.AgentApiUsageTrimmer"
+ "metabase_enterprise.api_keys.task.api_key_usage_trimmer.ApiKeyUsageTrimmer"
+ "metabase_enterprise.cache.task.refresh_cache_configs.Cache"
+ "metabase_enterprise.content_diagnostics.task.finding_trimmer.ContentDiagnosticsFindingTrimmer"
+ "metabase_enterprise.content_diagnostics.task.scan.ContentDiagnosticsScan"
+ "metabase_enterprise.data_complexity_score.task.complexity_score.DataComplexityScoring"
+ "metabase_enterprise.data_complexity_score.task.complexity_score_trimmer.DataComplexityScoreTrimmer"
+ "metabase_enterprise.dependencies.task.backfill.BackfillDependencies"
+ "metabase_enterprise.dependencies.task.entity_check.DependencyEntityCheck"
+ "metabase_enterprise.entity_retrieval.task.sync.OsiAiContextSync"
+ "metabase_enterprise.mcp.task.mcp_usage_trimmer.McpUsageTrimmer"
+ "metabase_enterprise.metabot.task.ai_usage_trimmer.AiUsageTrimmer"
+ "metabase_enterprise.remote_sync.task.import.AutoImport"
+ "metabase_enterprise.remote_sync.task.table_cleanup.RemoteSyncTableCleanup"
+ "metabase_enterprise.replacement.timeout.TimeoutReplacementRuns"
+ "metabase_enterprise.security_center.task.sync_advisories.SyncAdvisories"
+ "metabase_enterprise.semantic_search.task.index_cleanup.SemanticIndexCleanup"
+ "metabase_enterprise.semantic_search.task.index_repair.SemanticIndexRepair"
+ "metabase_enterprise.semantic_search.task.indexer.SemanticSearchIndexer"
+ "metabase_enterprise.semantic_search.task.metric_collector.SemanticMetricCollector"
+ "metabase_enterprise.semantic_search.task.usage_trimmer.SemanticSearchUsageTrimmer"
+ "metabase_enterprise.sso.task.delete_expired_relay_state.DeleteExpiredSsoRelayState"
+ "metabase_enterprise.support_access_grants.task.expire_grants.ExpireSupportAccessGrants"})
+
+(defn- job-namespaces
+ "Source namespaces that mention Quartz, which include every namespace that defines a job class."
+ []
+ (map :namespace (job-source-files)))
+
+(defn- defined-in-clojure?
+ "Whether Clojure source defines the class `c`, as a `deftype`, a `defrecord` or a `task/defjob` does."
+ [^Class c]
+ (or (.isAssignableFrom clojure.lang.IType c)
+ (.isAssignableFrom clojure.lang.IRecord c)))
+
+(defn- job-class-names
+ "The name of every job class defined in the source that Quartz can store."
+ []
+ ;; A namespace imports each class that it defines, whatever the class is named.
+ ;; The classes that Clojure source defines are ours, and the rest come from Quartz itself.
+ (into (sorted-set)
+ (for [ns-symb (job-namespaces)
+ :let [_ (classloader/require ns-symb)]
+ ^Class c (vals (ns-imports ns-symb))
+ :when (and (.isAssignableFrom org.quartz.Job c)
+ (defined-in-clojure? c)
+ ;; stands in for a job whose class is missing, so it is never stored
+ (not= (.getName c) "metabase.task.job_factory.NoOpJob"))]
+ (.getName c))))
+
+(deftest no-two-jobs-declare-the-same-saved-class-test
+ (is (= {}
+ (-> (group-by :saved-class @job-definitions)
+ (update-vals #(mapv (juxt :namespace :type-name) %))
+ (->> (into {} (filter #(> (count (val %)) 1))))))
+ (str "These jobs declare the same `:saved-class`. Quartz finds a job's class by that name, so the job"
+ " that loads last would run for all of them. Give each job its own name.")))
+
+(deftest every-job-class-declares-its-saved-class-test
+ (let [job-classes (job-class-names)
+ saved-classes (into #{} (map :saved-class) @job-definitions)]
+ (is (= {:job-classes-with-no-saved-class #{}
+ :saved-classes-with-no-job-class #{}}
+ {:job-classes-with-no-saved-class (set/difference job-classes saved-classes)
+ :saved-classes-with-no-job-class (set/difference saved-classes job-classes)})
+ (str "Define each job class with `task/defjob` or `task/defjob-type`. They take the `:saved-class`"
+ " that keeps the class name the same when the job moves. A plain `deftype` or `defrecord`"
+ " is named after its namespace."))))
+
+(deftest every-job-class-is-listed-test
+ (let [current (job-class-names)
+ listed (into #{} (filter in-this-edition?) listed-job-classes)]
+ (is (= {:not-listed #{}
+ :listed-but-gone #{}}
+ {:not-listed (set/difference current listed)
+ :listed-but-gone (set/difference listed current)})
+ (str "If you only moved or renamed a namespace or a type, or a bulk rename reached a `:saved-class`:"
+ " put the job's `:saved-class` back. A stored job is found by that name, so it stays the same"
+ " through a move.\n"
+ "For a new job: add its `:saved-class` to `listed-job-classes`.\n"
+ "For a job that was removed: remove its name.\n"
+ "To change a stored class name on purpose: change it in `listed-job-classes` too. Jobs stored"
+ " under the old name are deleted at the next startup, with their triggers."))))
diff --git a/test/metabase/task_test.clj b/test/metabase/task_test.clj
index a53fd6823e2..04a6d721cb6 100644
--- a/test/metabase/task_test.clj
+++ b/test/metabase/task_test.clj
@@ -6,7 +6,9 @@
[clojurewerkz.quartzite.scheduler :as qs]
[clojurewerkz.quartzite.triggers :as triggers]
[metabase.app-db.connection :as mdb.connection]
+ [metabase.classloader.core :as classloader]
[metabase.task.core :as task]
+ [metabase.task.impl :as task.impl]
[metabase.test :as mt]
[metabase.test.fixtures :as fixtures]
[metabase.test.util :as tu]
@@ -14,7 +16,10 @@
[metabase.util.malli.schema :as ms]
[toucan2.core :as t2])
(:import
- (org.quartz CronTrigger JobDetail)))
+ (java.lang.annotation Annotation)
+ (java.time Duration Instant)
+ (java.util Date)
+ (org.quartz CronTrigger InterruptableJob Job JobDetail)))
(set! *warn-on-reflection* true)
@@ -22,7 +27,9 @@
;; make sure we attempt to reschedule tasks so changes made in source are propogated to JDBC backend
-(task/defjob TestJob [_])
+(task/defjob TestJob
+ {:saved-class "metabase.task_test.TestJob"}
+ [_])
(defn- job ^JobDetail []
(jobs/build
@@ -161,3 +168,183 @@
(if scheduler-initialized?
(task/start-scheduler!)
(task/stop-scheduler!))))))
+
+(defmacro ^:private with-jdbc-scheduler!
+ "Runs `body` with a JDBC-backed scheduler, and stops it afterwards unless one was already running.
+ A scheduler that this starts stays in standby: it stores and loads jobs, and fires no trigger."
+ [& body]
+ `(let [running?# (some? (#'task/scheduler))]
+ (when-not running?#
+ ;; the tasks' initializers start threads that outlive the scheduler, so only the scheduler starts
+ (mt/with-dynamic-fn-redefs [task.impl/init-tasks! (constantly nil)]
+ (task/init-scheduler!)))
+ (try
+ ~@body
+ (finally
+ (when-not running?#
+ (task/stop-scheduler!))))))
+
+(defn- stored-job-class-name []
+ (t2/select-one-fn (capitalize-if-mysql :job_class_name)
+ (capitalize-if-mysql :qrtz_job_details)
+ (capitalize-if-mysql :job_name) "metabase.task-test.job"))
+
+(defn- trigger-that-starts-next-week
+ "An hourly trigger like [[trigger-1]] that can't fire while a test runs."
+ ^CronTrigger []
+ (triggers/build
+ (triggers/with-identity (triggers/key "metabase.task-test.trigger"))
+ (triggers/start-at (Date/from (.plus (Instant/now) (Duration/ofDays 7))))
+ (triggers/with-schedule
+ (cron/schedule
+ (cron/cron-schedule "0 0 * * * ? *")
+ (cron/with-misfire-handling-instruction-do-nothing)))))
+
+(defonce ^:private moved-job-runs (atom 0))
+
+(task/defjob MovedJob
+ "A job as it is after its namespace moved: its saved class name is not the one this namespace would give."
+ {:saved-class "metabase.task_test_before_the_move.MovedJob"}
+ [_]
+ (swap! moved-job-runs inc))
+
+(defn- moved-job ^JobDetail []
+ (jobs/build
+ (jobs/of-type MovedJob)
+ (jobs/with-identity (jobs/key "metabase.task-test.job"))))
+
+(defn- loadable-class-name
+ "Returns `class-name` if a class of that name loads, as Quartz would load a stored job's class."
+ [class-name]
+ (try
+ (.getName (Class/forName class-name true (classloader/the-classloader)))
+ (catch ClassNotFoundException _
+ nil)))
+
+(deftest startup-cleanup-keeps-a-moved-job-that-kept-its-saved-class-test
+ ;; The row is stored under the `:saved-class`, which is not the name the namespace would give the class.
+ (with-jdbc-scheduler!
+ (try
+ ;; the row is this test's own, and its trigger starts next week, so the scheduler never fires it
+ (task/schedule-task! (moved-job) (trigger-that-starts-next-week))
+ (#'task.impl/delete-jobs-with-no-class!)
+ (let [^JobDetail stored (qs/get-job (#'task/scheduler) (.getKey (moved-job)))
+ ^Class job-class (.getJobClass stored)
+ runs-before @moved-job-runs]
+ ;; Quartz fires a job by calling `execute` on a new instance of the class that the stored name loads.
+ (.execute ^Job (.newInstance (.getDeclaredConstructor job-class (make-array Class 0))
+ (object-array 0))
+ nil)
+ (is (= {:stored-class-name "metabase.task_test_before_the_move.MovedJob"
+ :loaded-class-name "metabase.task_test_before_the_move.MovedJob"
+ :defined-here? true
+ :name-from-namespace nil
+ :triggers #{{:cron-expression "0 0 * * * ? *"
+ :misfire-instruction CronTrigger/MISFIRE_INSTRUCTION_DO_NOTHING}}
+ :runs 1}
+ {:stored-class-name (stored-job-class-name)
+ :loaded-class-name (.getName job-class)
+ :defined-here? (identical? job-class MovedJob)
+ :name-from-namespace (loadable-class-name "metabase.task_test.MovedJob")
+ :triggers (triggers)
+ :runs (- @moved-job-runs runs-before)})))
+ (finally
+ (task/delete-task! (.getKey (moved-job)) (.getKey (trigger-1)))))))
+
+(task/defjob AnnotatedJob
+ "A job with both Quartz annotations."
+ {:saved-class "metabase.task_test.AnnotatedJob"
+ :concurrent? false
+ :persist-data? true}
+ [_])
+
+(task/defjob DefaultsJob
+ {:saved-class "metabase.task_test.DefaultsJob", :concurrent? true, :persist-data? false}
+ [_])
+
+(task/defjob-type InterruptableTestJob
+ "A job that implements more than `org.quartz.Job`."
+ {:saved-class "metabase.task_test.InterruptableTestJob", :concurrent? false}
+ org.quartz.Job
+ (execute [_ _])
+ org.quartz.InterruptableJob
+ (interrupt [_]))
+
+(defn- annotation-names [^Class c]
+ (into #{} (map #(.getSimpleName (.annotationType ^Annotation %))) (.getAnnotations c)))
+
+(deftest defjob-options-put-quartz-annotations-on-the-class-test
+ (is (= {:annotated {:class-name "metabase.task_test.AnnotatedJob"
+ :annotations #{"DisallowConcurrentExecution"
+ "PersistJobDataAfterExecution"}
+ :concurrent-execution-disallowed? true
+ :persist-job-data? true}
+ :defaults {:class-name "metabase.task_test.DefaultsJob"
+ :annotations #{}
+ :concurrent-execution-disallowed? false
+ :persist-job-data? false}
+ :interruptable {:class-name "metabase.task_test.InterruptableTestJob"
+ :annotations #{"DisallowConcurrentExecution"}
+ :concurrent-execution-disallowed? true
+ :persist-job-data? false}}
+ (update-vals {:annotated AnnotatedJob
+ :defaults DefaultsJob
+ :interruptable InterruptableTestJob}
+ (fn [^Class c]
+ (let [^JobDetail detail (jobs/build (jobs/of-type c))]
+ {:class-name (.getName c)
+ :annotations (annotation-names c)
+ :concurrent-execution-disallowed? (.isConcurrentExectionDisallowed detail)
+ :persist-job-data? (.isPersistJobDataAfterExecution detail)}))))))
+
+(deftest defjob-defines-a-factory-test
+ ;; Kondo lints `task/defjob` as a `defn`, so it does not know the factory of a job defined with it.
+ (let [->annotated-job (ns-resolve 'metabase.task-test '->AnnotatedJob)]
+ (is (= {:job? true
+ :interruptable? true
+ :docstring "A job with both Quartz annotations."}
+ {:job? (instance? Job (->annotated-job))
+ :interruptable? (instance? InterruptableJob (->InterruptableTestJob))
+ :docstring (:doc (meta ->annotated-job))}))))
+
+(defn- definition-error
+ "Returns the message of the error that expanding the job definition `form` in this namespace throws."
+ [form]
+ (binding [*ns* (the-ns 'metabase.task-test)]
+ (try
+ (when (macroexpand-1 form)
+ nil)
+ (catch Throwable e
+ (ex-message (or (ex-cause e) e))))))
+
+(deftest defjob-rejects-an-invalid-definition-test
+ (is (=? {;; the entry to paste, with the class name that this namespace gives the type
+ :no-saved-class #"(?s).*\n\n :saved-class \"metabase\.task_test\.NewJob\"\n\n.*"
+ :no-options #"(?s).*\n\n \{:saved-class \"metabase\.task_test\.NewJob\"\}\n"
+ :unknown-option #".*unknown options \[:durable\?\]\. The options are \[.*\]\."
+ :not-a-literal #".*its `:saved-class` must be a string literal\."
+ :not-a-class-name #".*\"metabase\.task-test\.NewJob\" is not a fully qualified Java class name.*"
+ :no-package #".*\"NewJob\" is not a fully qualified Java class name.*"
+ :not-a-boolean #".*its `:concurrent\?` must be `true` or `false`\."
+ :metadata #".*its type name has metadata\..*"
+ :two-arguments #".*its argument vector takes one binding.*"
+ :valid nil}
+ (update-vals
+ '{:no-saved-class (task/defjob NewJob {:concurrent? false} [_])
+ :no-options (task/defjob NewJob [_])
+ :unknown-option (task/defjob NewJob {:saved-class "a.NewJob", :durable? true} [_])
+ :not-a-literal (task/defjob NewJob {:saved-class (str "a." "NewJob")} [_])
+ :not-a-class-name (task/defjob NewJob {:saved-class "metabase.task-test.NewJob"} [_])
+ :no-package (task/defjob-type NewJob {:saved-class "NewJob"} org.quartz.Job)
+ :not-a-boolean (task/defjob NewJob {:saved-class "a.NewJob", :concurrent? nil} [_])
+ :metadata (task/defjob ^{:doc "A job."} NewJob {:saved-class "a.NewJob"} [_])
+ :two-arguments (task/defjob NewJob {:saved-class "a.NewJob"} [_ _])
+ :valid (task/defjob NewJob "A job." {:saved-class "a.NewJob"} [_])}
+ definition-error))))
+
+(deftest defjob-allows-the-position-metadata-of-a-reader-test
+ ;; Eastwood reads the source with a reader that puts the line and column on every symbol.
+ (is (nil? (definition-error (list 'task/defjob
+ (with-meta 'NewJob {:line 1, :column 14})
+ {:saved-class "a.NewJob"}
+ '[_])))))