Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.
Solid Queue as the Default Job Backend — Single-Container, Postgres-Only Deployment
Status: Implemented (starter + CLI + docs — spree-starter#1368, spree#14323)
Target: Spree 5.6 (spree-starter + tooling change only — no core gem dependency; can ship in a 5.6.x starter update if RC feedback warrants)
Depends on: Single-node Docker deployment (spree-starter#1368 + spree#14323, shipped); 6.0-stock-reservations.md (consumes the recurring-jobs mechanism this plan standardizes)
Author: Damian Legawiec (design discussion with Claude, 2026-07-18)
Last updated: 2026-07-18 (implementation landed same day; verified on the built image)
Summary
Replace Sidekiq + Redis with the Rails 8 Solid stack in spree-starter, making the default Spree deployment one process, one container, one datastore: Puma runs the Solid Queue supervisor in-process (plugin :solid_queue), jobs live in Postgres (which Spree already requires), Solid Cache replaces the Redis cache store, and Solid Cable replaces the Redis Action Cable adapter. Redis disappears from the default stack entirely — dev compose, production compose, and the Render Blueprint each lose a service. This is the "ONCE"/Rails 8 deployment model, and it beats the simplest stories in the category: Medusa's workerMode: "shared" still wants Redis in production; Vendure's DB-backed JobQueue is the same bet without the in-process default; Saleor has no single-process story at all. Scale-out is a config flag, not a code change.
Key Decisions (do not deviate without discussion)
- Solid Queue is the default adapter in spree-starter; Sidekiq remains fully supported. Core is adapter-agnostic (audited 2026-07-18 — see Design Details) and stays that way. Existing projects keep whatever their repo configures; nothing in the gems forces a migration.
- In-Puma supervisor is the default topology.
puma.rbgetsplugin :solid_queue if ENV.fetch("SOLID_QUEUE_IN_PUMA", "true") == "true". The escape hatch for scale isSOLID_QUEUE_IN_PUMA=falseon web plus a worker service runningbin/jobsfrom the same image — one env var and one service definition, no rebuild. - Single database by default; the queue-DB split is a documented scaling step, not the default. Single-DB gives commerce the property that matters:
perform_laterinside a transaction commits atomically with the business data ("order created → confirmation email" cannot race). The Rails 8 two-DB convention (spree_production_queue, same Postgres server) is documented for high-churn installs alongside the worker split. - Full no-Redis: Solid Cache for
Rails.cache, Solid Cable for Action Cable (the legacy admin's import live-feed broadcasts still need a cable adapter until the engine is removed in 6.0). Meilisearch stays — it is a search engine, not incidental infra. - Solid Cache lives in the primary database — no separate cache DB. Headless reframing: the Russian-doll fragment caching that justified Redis-sized caches died with the ERB storefront, and the legacy admin's partial caching dies in 6.0. What remains is ~20 low-level
Rails.cache.fetchsites (taxon descendant ids, states helper, the preferences store) plus rate-limit counters — a workload where splitting databases is over-engineering. Redis/Valkey stays a documented one-line swap (config.cache_store = :redis_cache_store) for high-traffic installs; wherever Redis remains in docs, say "Redis or Valkey" (post-relicensing, platforms steer to Valkey). It's a package deal — no half-migration that keeps Sidekiq while adopting Solid Cache; the payoff is deleting the Redis requirement entirely. - Mission Control – Jobs replaces the
/sidekiqUI (mission_control-jobs, mounted at/jobs). Guarded by Mission Control's own HTTP Basic auth (MISSION_CONTROL_USER/MISSION_CONTROL_PASSWORD, seed-admin defaults in dev/test, Render Blueprint generates the password) — deliberately not the admin Devise session, which6.0-platform-auth.mdremoves from the Rails app. - Recurring jobs standardize on Solid Queue's
config/recurring.yml— this is the mechanism6.0-stock-reservations.mdshould name forExpireJob(it currently hedges "sidekiq-cron / solid_queue").
Design Details
Core audit (2026-07-18) — why the gems need zero changes
- No Sidekiq-specific APIs anywhere in
spree/core,spree/api,spree/admin,spree/emails: noSidekiq::constants, no batches, no unique-jobs middleware, no server middleware. The only mentions are comments. - Scheduling is plain ActiveJob: five
set(wait: 2.seconds..30.seconds)sites (product taxon auto-match, import row fan-out, theme duplication, payment webhook retry). Solid Queue's dispatcher handles scheduled jobs natively at this granularity. Spree::BaseJob'sretry_on ActiveRecord::RecordNotFoundexists to absorb the enqueue-before-commit race — which single-DB Solid Queue eliminates (the enqueue commits with the transaction; Solid Queue ≥1.2 exposesenqueue_after_transaction_committo control this). The retry stays for adapter-agnosticism, but the default deployment no longer exercises it.- No Redis usage in core: the events system is explicitly adapter-abstracted; no advisory locks, no Redlock, no direct
Redis.new. - Action Cable surface:
Spree::Import+ the legacy admin's import subscribers broadcast Turbo Streams (live import feed). The React Dashboard polls instead (by design — see5.6-admin-spa-csv-import.md), so this coupling dies with the legacy admin in 6.0; until then Solid Cable carries it. - No
sidekiq-cron/schedule file exists today — there is no recurring-job migration, only a green field forrecurring.yml. - Extensions are clean too (confirmed by Damian, 2026-07-18): all extensions — including
spree_stripe,spree_adyen,spree_paypal_checkout— use plain ActiveJob with no Sidekiq-specific APIs. - Cache audit: 20
Rails.cachecall sites across core+api, all small-valuefetchmemoization — no fragment caching in the API surface. The highest-frequency cache operation is rate limiting: the v3 API uses Rails 8's built-inrate_limitwithstore: Rails.cacheon the base controller (per-key, every request) plus tighter limits on login/refresh/register/password-reset/newsletter/webhooks. Verified (2026-07-18): Solid Cache implements atomicincrement, and Rails'rate_limitwas deliberately built over the genericActiveSupport::Cachecontract (rails/rails#50781) — the combination is supported and correct across processes. Cost model: one PostgresUPDATEper API request; fine for the median install, and the swap knob exists for hot paths.
spree-starter changes
| File | Change |
|---|---|
Gemfile |
− sidekiq, sentry-sidekiq, redis; + solid_queue, solid_cache, solid_cable, mission_control-jobs (sentry-rails already instruments ActiveJob) |
config/application.rb |
config.active_job.queue_adapter = :solid_queue |
config/puma.rb |
plugin :solid_queue if ENV.fetch("SOLID_QUEUE_IN_PUMA", "true") == "true" |
config/environments/production.rb |
config.cache_store = :solid_cache_store (drop the REDIS_CACHE_URL branch); config.solid_queue.connects_to not set (single-DB default) |
config/cable.yml |
adapter: solid_cable (dev + production) |
config/database.yml |
Pool sizing comment + formula: Puma threads + Solid Queue worker threads + ~3 (dispatcher/scheduler/supervisor). This is the one place in-Puma mode bites people — pool exhaustion that presents as a Puma problem. |
config/routes.rb |
mount MissionControl::Jobs::Engine => "/jobs" behind admin auth, replacing Sidekiq::Web |
db/ |
Solid Queue/Cache/Cable schemas in the main migrations path (single-DB) |
config/recurring.yml |
Empty scaffold with a commented example (ready for stock-reservation expiry etc.) |
docker-compose.yml / docker-compose.dev.yml |
Drop the redis service and the worker service; drop REDIS_URL env; web is the only app container |
render.yaml |
Web service unchanged (Docker, repo-root context); redis service deleted; the commented worker block becomes the scale-out example: same image, dockerCommand: bundle exec bin/jobs, SOLID_QUEUE_IN_PUMA=false on web |
.env.example |
− REDIS_URL, SIDEKIQ_*; + SOLID_QUEUE_IN_PUMA (documented, default true) |
Deployment shapes
Default (create-spree-app, docker run, Render free tier):
┌─────────────────────────────┐ ┌──────────────┐
│ app container │ │ Postgres │
│ Puma │─────▶│ (one DB) │
│ └─ plugin :solid_queue │ └──────────────┘
│ ├─ dispatcher │
│ ├─ worker(s) │
│ └─ scheduler │
└─────────────────────────────┘
Scale-out (flag flip, same image):
web: SOLID_QUEUE_IN_PUMA=false → Puma only
worker: bundle exec bin/jobs → Solid Queue supervisor only
db: optionally spree_production_queue (second DB, same server)
via config.solid_queue.connects_to + db/queue_migrate
CLI / create-spree-app changes
spree devrunsdocker compose up web workertoday — with noworkerservice it must detect the compose shape (worker service present → include it; absent → web only). Same forprimeBundleVolume's assumptions andspree logs worker(error message pointing at Mission Control /SOLID_QUEUE_IN_PUMAinstead).- Setup summary/README/docs drop the Sidekiq UI link; add
/jobs(Mission Control). SIDEKIQ_DB_POOLenv handling removed; pool guidance moves to the database.yml comment.- Generated project docs (README/CLAUDE.md templates) describe the single-process model and the two scale-out steps.
Competitor context (for docs/marketing framing)
- Medusa:
workerMode: "server" | "worker" | "shared"— shared in one process for dev/simple deploys, but production guidance splits and the event bus/queue want Redis. Spree's story: same simplicity, Postgres-only. - Vendure: explicit
bootstrap()vsbootstrapWorker()split; default JobQueue is DB-backed (same bet), BullMQ/Redis as the production-grade plugin. - Saleor: Django + Celery + Redis broker, mandatory separate workers — no single-process story.
Migration Path
- spree-starter PR: the table above, in one change. CI: boot + a job-exercising smoke (e.g. seed triggers
AutoMatchTaxonsJob; assert it executes in-Puma). - CLI + create-spree-app PR: compose-shape detection in
spree dev/logs, template/doc updates. Backward-compatible — existing Sidekiq-shaped projects keep working (worker service detected and included). - Docs: deployment guide gains the two scale-out steps (worker split, queue-DB split); environment-variables page updated.
- Existing projects: no forced migration. A docs recipe covers switching (add gems, flip adapter, drain Sidekiq queues before cutover — Sidekiq jobs don't transfer).
- 6.0:
6.0-stock-reservations.mdnamesrecurring.ymlforExpireJob; legacy-admin removal deletes the last Action Cable broadcasts (Solid Cable stays for storefront/custom use).
Constraints on Current Work
- Keep core adapter-agnostic: no Sidekiq-specific APIs (batches, unique-jobs, middleware) and no direct Redis usage in any Spree gem. New jobs inherit
Spree::BaseJoband use plain ActiveJob scheduling (set(wait:)). - New recurring-work designs should name
config/recurring.ymlas the mechanism, not sidekiq-cron/whenever. - Don't add Redis-backed features to the starter (e.g. Rack::Attack with a Redis store) without routing them through
Rails.cache. - CLI work touching
spree dev/compose should not hard-code theworkerservice name — detect it.