Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.
Ruby b2b-commercee-commerceecommerceecommerce-apiecommerce-frameworkecommerce-platformheadlessheadless-commerceheadless-ecommercemarketplacemulti-tenantmulti-vendormulti-vendor-ecommerceopen-sourcespree-commerce
Never lose an earning to a seller who is not payable yet
Damian Legawiec951380b5b329
diff --git a/spree/api/app/controllers/spree/api/v3/admin/seller_payouts_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/seller_payouts_controller.rbindex f5b7e053a7..00a08bf89e 100644--- a/spree/api/app/controllers/spree/api/v3/admin/seller_payouts_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/seller_payouts_controller.rb@@ -45,8 +45,10 @@ module Spreesuper.for_store(current_store)end+ # Deliberately without `transfers`: the serializer wants their number,+ # not the rows, and a monthly settlement can hold hundreds.def collection_includes- [:seller, :transfers]+ [:seller]endendenddiff --git a/spree/api/app/controllers/spree/api/v3/admin/seller_transfers_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/seller_transfers_controller.rbindex 7453aeae30..49c1ded90e 100644--- a/spree/api/app/controllers/spree/api/v3/admin/seller_transfers_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/seller_transfers_controller.rb@@ -24,8 +24,10 @@ module Spreesuper.for_store(current_store)end+ # Every association the serializer reads, so a page of rows costs a+ # fixed number of queries rather than one per row.def collection_includes- [:seller, :order, :payout]+ [:seller, :order, :payout, :reversed_from, :refund]endendenddiff --git a/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rbindex 9e40ff1d23..71cac40ec5 100644--- a/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rb@@ -31,7 +31,7 @@ module Spree# How many earnings this settlement covers, so a list answers "what# is in this deposit" without loading every transfer.- attribute(:transfers_count) { |payout| payout.transfers.size }+ attribute(:transfers_count) { |payout| payout.transfers.count }endendenddiff --git a/spree/core/app/jobs/spree/seller_transfers/execute_pending_job.rb b/spree/core/app/jobs/spree/seller_transfers/execute_pending_job.rbnew file mode 100644index 0000000000..499cd12333--- /dev/null+++ b/spree/core/app/jobs/spree/seller_transfers/execute_pending_job.rb@@ -0,0 +1,33 @@+module Spree+ module SellerTransfers+ # Asks the provider again about earnings it could not be asked about before.+ #+ # A seller who ships while their payout account is still being verified has+ # earned all the same, so the ledger records the row and leaves it pending.+ # What clears it is this job, run when the seller becomes payable — the+ # event that changed, rather than the next scheduled settlement.+ #+ # That distinction matters: a sweep would never reach these rows, because+ # it looks for confirmed earnings and a seller whose only rows are pending+ # has none, so nothing would prompt it in the one case this exists for.+ class ExecutePendingJob < ::Spree::BaseJob+ queue_as Spree.queues.payouts++ # @param seller_id [Integer]+ def perform(seller_id)+ seller = Spree::Seller.find_by(id: seller_id)+ return if seller.nil? || !seller.payouts_enabled?++ provider = seller.store.payout_provider_instance++ seller.seller_transfers.pending.where(payout_id: nil).find_each do |transfer|+ provider.transfer!(transfer)+ rescue StandardError => e+ # One seller's stuck earning must not stop the rest — the row stays+ # pending and the next attempt finds it.+ Rails.error.report(e, handled: true, context: { seller_transfer_id: transfer.id }, source: 'spree.core')+ end+ end+ end+ end+enddiff --git a/spree/core/app/models/spree/seller.rb b/spree/core/app/models/spree/seller.rbindex e842327d00..9b00a7418a 100644--- a/spree/core/app/models/spree/seller.rb+++ b/spree/core/app/models/spree/seller.rb@@ -238,6 +238,8 @@ module Spree## @return [Boolean]def payouts_enabled?+ return true unless store.payout_provider_class.requires_payout_account?+payouts_enabled_at.present?enddiff --git a/spree/core/app/models/spree/seller_payout.rb b/spree/core/app/models/spree/seller_payout.rbindex 1ba26b0100..b41e01635d 100644--- a/spree/core/app/models/spree/seller_payout.rb+++ b/spree/core/app/models/spree/seller_payout.rb@@ -68,5 +68,30 @@ module Spreedef transfers_totaltransfers.sum(:amount)end++ # Gives up on this settlement and puts its earnings back.+ #+ # Releasing them is what makes a failure recoverable, and it is the one+ # thing a caller must not forget: a sweep only ever collects unstamped+ # rows, so earnings left stamped to a failed payout are invisible to every+ # later sweep while the balance still reports the seller as owed them.+ # Released, they simply fall into the next one.+ #+ # Lives here rather than in the flows that call it — core's sweep and any+ # provider's failure webhook — because the association and the status both+ # belong to the payout, and a provider gem should not have to know the+ # invariant to honour it.+ def fail!+ transaction do+ release_transfers+ update!(status: 'failed', amount: 0)+ end+ end++ # Hands the earnings back to the next sweep without recording a failure —+ # for a settlement that never had anything to send.+ def release_transfers+ transfers.update_all(payout_id: nil, updated_at: Time.current)+ endendenddiff --git a/spree/core/app/models/spree/store.rb b/spree/core/app/models/spree/store.rbindex 29390e3d87..5dd0c4f645 100644--- a/spree/core/app/models/spree/store.rb+++ b/spree/core/app/models/spree/store.rb@@ -84,6 +84,8 @@ module Spree# Monthly by default because it is the interval that needs least of an# operator paying by hand, which is what the built-in provider expects.preference :default_payouts_schedule_interval, :string, default: 'monthly'+ validates :preferred_default_payouts_schedule_interval,+ inclusion: { in: Spree::Seller::PAYOUT_INTERVALS }, allow_blank: true# What a seller's balance must reach before a settlement is worth sending;# below it the balance carries to the next period. Zero pays whatever is# owed, which is right for a provider that moves money for free.@@ -392,13 +394,31 @@ module Spree# Who pays this store's sellers, ready to be asked.#- # Resolved per call rather than memoized, so a marketplace that connects a- # provider mid-process starts using it without a restart — the same shape- # markets use for tax.+ # Who moves money to this store's sellers.+ #+ # Memoized per record, not per process: a store reloaded after an operator+ # changes the setting resolves afresh, so a marketplace that connects a+ # provider mid-process still starts using it without a restart.## @return [Spree::PayoutProvider::Base]def payout_provider_instance- (preferred_payout_provider.presence || Spree.default_payout_provider.to_s).constantize.new+ @payout_provider_instance ||= payout_provider_class.new+ end++ # The class alone, for the questions that are about the provider rather+ # than about a movement — there is no need to build one to ask what it+ # requires.+ #+ # @return [Class]+ def payout_provider_class+ configured = preferred_payout_provider.presence+ # A name no longer in the registry — a typo, or a gem since removed —+ # would otherwise raise inside a subscriber on every fulfillment and stop+ # the ledger recording anything. The built-in provider keeps the books+ # until an operator fixes the setting.+ configured = nil unless Spree.payout_providers.any? { |provider| provider.to_s == configured }++ (configured || Spree.default_payout_provider.to_s).constantizeenddef formatted_urldiff --git a/spree/core/app/workflows/spree/seller_payouts/sweep.rb b/spree/core/app/workflows/spree/seller_payouts/sweep.rbindex 5dea4d8b81..a325dc09ed 100644--- a/spree/core/app/workflows/spree/seller_payouts/sweep.rb+++ b/spree/core/app/workflows/spree/seller_payouts/sweep.rb@@ -45,8 +45,6 @@ module Spree# The earnings simply stay unsettled until one exists — they are not# lost, and the next sweep will find them.def ensure_payable- return unless provider.class.requires_payout_account?-halt!(seller) unless seller.payouts_enabled?end@@ -99,14 +97,14 @@ module Spreeenddef restate_amount- @payout.update!(amount: @payout.transfers.sum(:amount))+ @payout.update!(amount: @payout.transfers_total)end# Nothing was claimed, so there is nothing to release and nothing to# record. The earnings this sweep meant to take are already in the payout# that won them.def discard_empty_payout- @payout.transfers.update_all(payout_id: nil, updated_at: Time.current)+ @payout.release_transfers@payout.destroy!halt!(seller)end@@ -114,17 +112,7 @@ module Spreedef execute_payoutprovider.pay!(payout)rescue StandardError => e- # Releasing the transfers is what makes a failure recoverable. A payout- # is created by the sweep and by nothing else, so transfers left- # stamped to a failed one are unreachable: the next sweep skips them- # (`unsettled` means unstamped) while the balance still reports them- # owed, and the seller is never paid. Released, they simply fall into- # the next sweep — which is the retry.- payout.transaction do- payout.transfers.update_all(payout_id: nil, updated_at: Time.current)- payout.update!(status: 'failed', amount: 0)- end-+ payout.fail!Rails.error.report(e, handled: true, context: { seller_payout_id: payout.id }, source: 'spree.core')failure(payout, e.message)end@@ -134,7 +122,7 @@ module Spreeenddef provider_name- provider.class.name.demodulize.underscore+ provider.class.provider_keyendendenddiff --git a/spree/core/app/workflows/spree/seller_transfers/create.rb b/spree/core/app/workflows/spree/seller_transfers/create.rbindex 2ed01ea725..c7aecb76db 100644--- a/spree/core/app/workflows/spree/seller_transfers/create.rb+++ b/spree/core/app/workflows/spree/seller_transfers/create.rb@@ -43,7 +43,6 @@ module Spreehalt!(order) if order.seller_id.blank?halt!(order) if order.fulfillments.empty?halt!(order) unless order.fully_fulfilled?- halt!(order) unless payable_seller?enddef replay_existing@@ -71,6 +70,12 @@ module Spree# Outside any transaction: a provider that moves money makes a network# call here, and a row lock must not be held across it.def execute_transfer+ # A seller the provider cannot pay yet keeps the earning as a pending+ # row. Verification can take days, and an account can lose the+ # capability again — either way the money is owed, and+ # `SellerTransfers::ExecutePendingJob` sends it once they are payable.+ return unless order.seller.payouts_enabled?+provider.transfer!(seller_transfer)rescue StandardError => e# The money may or may not have moved — that is precisely what an@@ -94,20 +99,12 @@ module Spree[base - order.commission_lines.sum(:total).to_d, 0].maxend- # A provider that executes transfers cannot pay a seller it has no- # account for — crediting them would promise money nothing can send.- def payable_seller?- return true unless provider.class.requires_payout_account?-- order.seller.payouts_enabled?- end-def provider@provider ||= order.store.payout_provider_instanceenddef provider_name- provider.class.name.demodulize.underscore+ provider.class.provider_keyendendenddiff --git a/spree/core/spec/jobs/spree/seller_transfers/execute_pending_job_spec.rb b/spree/core/spec/jobs/spree/seller_transfers/execute_pending_job_spec.rbnew file mode 100644index 0000000000..22d24897cd--- /dev/null+++ b/spree/core/spec/jobs/spree/seller_transfers/execute_pending_job_spec.rb@@ -0,0 +1,60 @@+require 'spec_helper'++RSpec.describe Spree::SellerTransfers::ExecutePendingJob do+ let(:store) { @default_store }+ let(:seller) { create(:seller, :approved, store: store, payouts_enabled_at: Time.current) }++ def pending_earning(amount = 40)+ create(:seller_transfer, seller: seller, amount: amount, status: 'pending',+ order: create(:order, store: store, seller: seller))+ end++ it 'sends what the provider could not be asked about before' do+ transfer = pending_earning++ described_class.perform_now(seller.id)++ expect(transfer.reload).to be_completed+ end++ it 'counts toward the balance once sent, so the next sweep settles it' do+ pending_earning++ expect { described_class.perform_now(seller.id) }.to change { seller.balance('USD') }.from(0).to(40)+ end++ it 'leaves an earning a settlement has already claimed' do+ transfer = pending_earning+ transfer.update!(payout: create(:seller_payout, seller: seller))++ described_class.perform_now(seller.id)++ expect(transfer.reload).to be_pending+ end++ # One seller's stuck earning must not stop the rest.+ it 'carries on past an earning the provider refuses' do+ first = pending_earning(40)+ second = pending_earning(30)+ allow_any_instance_of(Spree::PayoutProvider::System).to receive(:transfer!).and_wrap_original do |original, transfer|+ raise StandardError, 'provider down' if transfer.id == first.id++ original.call(transfer)+ end++ described_class.perform_now(seller.id)++ expect(first.reload).to be_pending+ expect(second.reload).to be_completed+ end++ it 'does nothing for a seller the provider still will not accept' do+ allow(Spree::PayoutProvider::System).to receive(:requires_payout_account?).and_return(true)+ seller.update!(payouts_enabled_at: nil)+ transfer = pending_earning++ described_class.perform_now(seller.id)++ expect(transfer.reload).to be_pending+ end+enddiff --git a/spree/core/spec/workflows/spree/seller_payouts/sweep_spec.rb b/spree/core/spec/workflows/spree/seller_payouts/sweep_spec.rbindex 02026be252..ba79df5b22 100644--- a/spree/core/spec/workflows/spree/seller_payouts/sweep_spec.rb+++ b/spree/core/spec/workflows/spree/seller_payouts/sweep_spec.rb@@ -47,7 +47,10 @@ RSpec.describe Spree::SellerPayouts::Sweep doexpect(described_class.call(seller: seller, currency: 'USD').value.amount).to eq(30)end- it 'ignores an earning that is not yet confirmed' do+ # A pending earning is one the provider has not confirmed. Clearing those+ # is `SellerTransfers::ExecutePendingJob`'s job, driven by the seller+ # becoming payable rather than by the calendar.+ it 'leaves an earning the provider has not confirmed' doearn(40)earn(30, status: 'pending')diff --git a/spree/core/spec/workflows/spree/seller_transfers/create_spec.rb b/spree/core/spec/workflows/spree/seller_transfers/create_spec.rbindex de7d9e7503..db64a0a9f9 100644--- a/spree/core/spec/workflows/spree/seller_transfers/create_spec.rb+++ b/spree/core/spec/workflows/spree/seller_transfers/create_spec.rb@@ -79,6 +79,30 @@ RSpec.describe Spree::SellerTransfers::Create doendend+ # A seller whose account is still being verified has earned all the same.+ # `order.fulfilled` fires once and nothing re-drives it, so refusing to+ # write the row would lose the earning outright.+ describe 'when the provider cannot pay the seller yet' do+ before do+ allow_any_instance_of(Spree::PayoutProvider::System).to receive(:requires_payout_account?).and_return(true)+ allow(Spree::PayoutProvider::System).to receive(:requires_payout_account?).and_return(true)+ end++ it 'still records what they earned' do+ expect { described_class.call(order: shipped_order) }.to change { Spree::SellerTransfer.count }.by(1)+ end++ it 'leaves it pending rather than claiming the money moved' do+ expect(described_class.call(order: shipped_order).value).to be_pending+ end++ it 'never asks the provider to send it' do+ expect_any_instance_of(Spree::PayoutProvider::System).not_to receive(:transfer!)++ described_class.call(order: shipped_order)+ end+ end+describe 'replay' doit 'returns the earning that exists rather than crediting twice' doorder = shipped_order@@ -96,7 +120,7 @@ RSpec.describe Spree::SellerTransfers::Create doresult = described_class.call(order: shipped_order)expect(result.value).to be_completed- expect(result.value.provider).to eq('system')+ expect(result.value.provider).to eq('Spree::PayoutProvider::System')endit 'counts toward the seller’s balance' dodiff --git a/spree/providers/stripe/app/models/spree_stripe/gateway/connect.rb b/spree/providers/stripe/app/models/spree_stripe/gateway/connect.rbindex e03d12e94b..ad598204ed 100644--- a/spree/providers/stripe/app/models/spree_stripe/gateway/connect.rb+++ b/spree/providers/stripe/app/models/spree_stripe/gateway/connect.rb@@ -76,11 +76,6 @@ module SpreeStripe).urlend- # @return [Hash] what every Stripe call from this gateway is signed with- def api_options- { api_key: preferred_secret_key }- end-# Where Stripe sends events originating in sellers' connected accounts.# @return [String, nil]def connect_webhook_url@@ -124,27 +119,41 @@ module SpreeStripeseller = store.sellers.find_by(payout_account_reference: account.id)return if seller.nil?+ became_payable = account.payouts_enabled && seller.payouts_enabled_at.nil?seller.update!(payouts_enabled_at: account.payouts_enabled ? (seller.payouts_enabled_at || Time.current) : nil)++ # Anything they earned while unverified is owed and still pending, and+ # this is the moment it can finally be sent.+ Spree::SellerTransfers::ExecutePendingJob.perform_later(seller.id) if became_payableend# Stripe's payout belongs to a connected account rather than to one of- # our settlements, so the seller is found by the account the event names- # and the settlement by being the one still owed in that currency.+ # our settlements, so the seller is found by the account the event names.+ #+ # Matching the settlement is by Stripe's own id first. Stripe redelivers+ # a webhook on any non-2xx or timeout, and without that match a second+ # delivery would skip the settlement it already completed and land on the+ # next one still owed — marking an unrelated payout paid.def handle_payout(event, status)object = event.data.objectseller = store.sellers.find_by(payout_account_reference: event.account)return if seller.nil?- payout = seller.seller_payouts.owed.where(currency: object.currency.to_s.upcase).order(:created_at).first- return if payout.nil?+ payout = find_payout(seller, object)+ return if payout.nil? || payout.completed?if status == 'paid'Spree.seller_payout_complete_workflow.call(seller_payout: payout, reference: object.id)else- payout.update!(status: 'failed')+ payout.fail!endend+ def find_payout(seller, object)+ seller.seller_payouts.find_by(reference: object.id) ||+ seller.seller_payouts.owed.where(currency: object.currency.to_s.upcase).order(:created_at).first+ end+def verify_connect_webhook_signature(raw_body, headers)signature = headers['HTTP_STRIPE_SIGNATURE']@@ -167,13 +176,16 @@ module SpreeStripesecrets.select(&:present?)end- # Only registered once the store actually has sellers: an ordinary- # merchant account has no connected accounts to hear about, and the- # endpoint would receive nothing. Also the loop guard, as on the payment- # endpoint — registration writes the secret back through `update!`.+ # The stored secret is the loop guard, as on the payment endpoint —+ # registration writes it back through `update!`, which re-runs this.+ #+ # Deliberately not gated on the store having sellers. The ordinary setup+ # order is to connect Stripe first and invite sellers afterwards, and+ # nothing about creating a seller saves the gateway — so that guard meant+ # the endpoint was never registered at all, and `account.updated` never+ # arrived to say who could be paid.def create_connect_webhook_endpoint_asyncreturn if preferred_connect_webhook_signing_secret.present?- return unless store&.sellers&.exists?SpreeStripe::CreateWebhookEndpointJob.perform_later(id, connect: true)enddiff --git a/spree/providers/stripe/app/models/spree_stripe/gateway/webhooks.rb b/spree/providers/stripe/app/models/spree_stripe/gateway/webhooks.rbindex 60d98bcf76..580e130d05 100644--- a/spree/providers/stripe/app/models/spree_stripe/gateway/webhooks.rb+++ b/spree/providers/stripe/app/models/spree_stripe/gateway/webhooks.rb@@ -96,7 +96,7 @@ module SpreeStripedef create_webhook_endpoint_asyncreturn if preferred_webhook_signing_secret.present?- SpreeStripe::CreateWebhookEndpointJob.perform_later(id)+ SpreeStripe::CreateWebhookEndpointJob.perform_later(id, connect: false)endendenddiff --git a/spree/providers/stripe/spec/models/spree_stripe/gateway/connect_spec.rb b/spree/providers/stripe/spec/models/spree_stripe/gateway/connect_spec.rbindex cf2417e74a..4c524ba4e3 100644--- a/spree/providers/stripe/spec/models/spree_stripe/gateway/connect_spec.rb+++ b/spree/providers/stripe/spec/models/spree_stripe/gateway/connect_spec.rb@@ -86,15 +86,51 @@ RSpec.describe SpreeStripe::Gateway::Connect docontext 'payout.failed' dolet!(:payout) { create(:seller_payout, seller: seller, currency: 'USD') }- it 'marks the settlement failed so it is visible rather than silently stuck' do+ before doallow(Stripe::Webhook).to receive(:construct_event).and_return(stripe_event('payout.failed', { id: 'po_1', currency: 'usd' }, account: 'acct_seller'))+ end+ it 'marks the settlement failed so it is visible rather than silently stuck' dogateway.handle_payout_webhook(raw_body, headers)expect(payout.reload).to be_failedend++ # Left stamped to a failed settlement the earnings are unreachable: the+ # sweep only ever collects unstamped rows, while the balance still says+ # the seller is owed them.+ it 'releases the earnings so the next sweep is the retry' do+ create(:seller_transfer, :completed, seller: seller, payout: payout, amount: 40,+ order: create(:order, store: store, seller: seller))++ gateway.handle_payout_webhook(raw_body, headers)++ expect(seller.seller_transfers.unsettled.sum(:amount)).to eq(40)+ end+ end++ # Stripe redelivers on any non-2xx or timeout. Without matching on its own+ # payout id, a second delivery would skip the settlement it already+ # completed and land on the next one still owed.+ context 'when payout.paid is delivered twice' do+ let!(:first) { create(:seller_payout, seller: seller, currency: 'USD', amount: 40) }+ let!(:second) { create(:seller_payout, seller: seller, currency: 'USD', amount: 60) }++ before do+ allow(Stripe::Webhook).to receive(:construct_event).and_return(+ stripe_event('payout.paid', { id: 'po_1', currency: 'usd' }, account: 'acct_seller')+ )+ end++ it 'leaves the settlement it did not name alone' do+ gateway.handle_payout_webhook(raw_body, headers)+ gateway.handle_payout_webhook(raw_body, headers)++ expect(first.reload).to be_completed+ expect(second.reload).to be_pending+ endendit 'refuses an unsigned report — it could otherwise mark a seller payable' dodiff --git a/spree/providers/stripe/spec/models/spree_stripe/gateway_spec.rb b/spree/providers/stripe/spec/models/spree_stripe/gateway_spec.rbindex 0a10e67a09..3a185b141e 100644--- a/spree/providers/stripe/spec/models/spree_stripe/gateway_spec.rb+++ b/spree/providers/stripe/spec/models/spree_stripe/gateway_spec.rb@@ -1102,16 +1102,31 @@ RSpec.describe SpreeStripe::Gateway doendend+ # Two endpoints, registered independently: the payment one for the+ # marketplace's own charges, the Connect one for events originating inside+ # sellers' accounts. Each holds its own signing secret, and each secret is+ # the guard against registering that endpoint twice.describe 'webhook endpoint registration' do- it 'enqueues the registration job on create' do- expect { gateway }.to have_enqueued_job(SpreeStripe::CreateWebhookEndpointJob)+ it 'enqueues the payment registration job on create' do+ expect { gateway }.to have_enqueued_job(SpreeStripe::CreateWebhookEndpointJob).with(anything, connect: false)end- context 'when the signing secret is already stored' do- it 'does not enqueue the registration job' do+ it 'enqueues the Connect registration job on create' do+ expect { gateway }.to have_enqueued_job(SpreeStripe::CreateWebhookEndpointJob).with(anything, connect: true)+ end++ context 'when the payment signing secret is already stored' do+ it 'does not register the payment endpoint again' do+ expect do+ create(:stripe_gateway, :with_webhook_signing_secret, store: store)+ end.not_to have_enqueued_job(SpreeStripe::CreateWebhookEndpointJob).with(anything, connect: false)+ end++ # The Connect endpoint has its own secret, so it is still outstanding.+ it 'still registers the Connect endpoint' doexpect docreate(:stripe_gateway, :with_webhook_signing_secret, store: store)- end.not_to have_enqueued_job(SpreeStripe::CreateWebhookEndpointJob)+ end.to have_enqueued_job(SpreeStripe::CreateWebhookEndpointJob).with(anything, connect: true)endendend