Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.
Ruby b2b-commercee-commerceecommerceecommerce-apiecommerce-frameworkecommerce-platformheadlessheadless-commerceheadless-ecommercemarketplacemulti-tenantmulti-vendormulti-vendor-ecommerceopen-sourcespree-commerce
Added the PayPal payment and webhook upgrade steps
Łukasz Adamczyk8a7b6ceb53b2
diff --git a/spree/core/lib/spree/upgrades/5_6_to_6_0/manifest.yml b/spree/core/lib/spree/upgrades/5_6_to_6_0/manifest.ymlindex 5a254ef39c..044f197fed 100644--- a/spree/core/lib/spree/upgrades/5_6_to_6_0/manifest.yml+++ b/spree/core/lib/spree/upgrades/5_6_to_6_0/manifest.yml@@ -661,6 +661,40 @@ steps:hold a secret are skipped, and the legacy tables stay in place as arollback path until they are dropped in 6.1.+ - id: migrate_paypal_payments+ name: "Give PayPal payments taken before 6.0 the PayPal order id"+ task_class: "SpreePaypalCheckout::MaintenanceTasks::MigratePayments"+ optional: true+ notes: |+ Only runs when spree_paypal_checkout is installed; skipped otherwise.++ The 0.x gem stored a PayPal payment's capture id as its response code.+ In 6.0 the response code is always the PayPal order id, which is how a+ payment session finds its payment, and the capture id lives in the+ payment's metadata. This step rewrites each older payment to that shape,+ finding its order through the payment session that took it or, for the+ older checkout flow, through spree_paypal_checkout_orders.++ Run it before dropping spree_paypal_checkout_orders: that table is the+ only link from those older payments to their PayPal order. A payment it+ cannot match is logged by id, counted, and left as it was; refund it in+ PayPal. Supports a dry run, and is idempotent.++ - id: register_paypal_webhooks+ name: "Register each PayPal payment method's webhook"+ task: "spree:upgrade:register_paypal_webhooks"+ optional: true+ notes: |+ Only runs when spree_paypal_checkout is installed; skipped otherwise.++ PayPal webhooks register themselves in 6.0 and are always verified with+ the webhook's id. This finds or creates the webhook for each PayPal+ method's URL, replaces the events of one created by hand under the 0.x+ guide, and stores its id. Until it runs, methods created before the+ upgrade reject every webhook delivery.++ Idempotent.+- id: backfill_import_export_tenancyname: "Place every import in a store, and a seller where one ran it"task: "spree:upgrade:backfill_import_export_tenancy"diff --git a/spree/providers/paypal/app/maintenance_tasks/spree_paypal_checkout/maintenance_tasks/migrate_payments.rb b/spree/providers/paypal/app/maintenance_tasks/spree_paypal_checkout/maintenance_tasks/migrate_payments.rbnew file mode 100644index 0000000000..af14327e76--- /dev/null+++ b/spree/providers/paypal/app/maintenance_tasks/spree_paypal_checkout/maintenance_tasks/migrate_payments.rb@@ -0,0 +1,100 @@+module SpreePaypalCheckout+ module MaintenanceTasks+ # Gives PayPal payments taken under the 5.x gem the 6.0 shape: the PayPal+ # order id as the response code, the key core links a session to its+ # payment by, and the capture id the 5.x gem stored there moved to+ # metadata['paypal_capture_id'], where refunds and the dashboard link read+ # it.+ #+ # The order id is found two ways, both from the 5.x gem's own records: the+ # session that took the payment holds the capture in its external data, and+ # a payment from the older non-session flow is matched through+ # spree_paypal_checkout_orders, which must therefore not be dropped first.+ # A payment matched by neither is logged and left as it was.+ class MigratePayments < Spree::MaintenanceTask+ LEGACY_ORDERS_TABLE = 'spree_paypal_checkout_orders'.freeze++ supports_dry_run++ def collection+ gateway_ids = Spree::PaymentMethod.unscoped.where(type: 'SpreePaypalCheckout::Gateway').select(:id)+ Spree::Payment.unscoped.where(payment_method_id: gateway_ids).where.not(response_code: [nil, '']).order(:id)+ end++ def process(payment)+ # Payments taken in 6.0, and those an earlier run converted, already+ # carry what PayPal issued.+ if payment.metadata['paypal_capture_id'].present? || payment.metadata['paypal_authorization_id'].present?+ return tally(:already_converted)+ end++ paypal_order_id = paypal_order_id_for(payment)+ if paypal_order_id.nil?+ Rails.logger.warn("[spree_paypal_checkout] migrate_paypal_payments: no PayPal order found for payment #{payment.id}")+ return tally(:unmatched)+ end+ return tally(:would_convert) if dry_run?++ payment.update_columns(+ response_code: paypal_order_id,+ metadata: payment.metadata.merge('paypal_capture_id' => payment.response_code),+ updated_at: Time.current+ )+ tally(:converted)+ rescue ActiveRecord::RecordNotUnique+ # Another payment of the same order already carries this PayPal order.+ Rails.logger.warn("[spree_paypal_checkout] migrate_paypal_payments: payment #{payment.id} duplicates another payment's PayPal order")+ tally(:unmatched)+ end++ private++ def paypal_order_id_for(payment)+ capture_id = payment.response_code+ session_order_id(payment, capture_id) || legacy_order_id(payment, capture_id)+ end++ # The 5.x session stored PayPal's whole capture response as its external+ # data, the order id as its external id.+ # Looked up within the payment's own order or cart, so each payment reads+ # a handful of rows.+ def session_order_id(payment, capture_id)+ owner = payment.order_id ? { order_id: payment.order_id } : { cart_id: payment.cart_id }+ return if owner.values.first.nil?++ sessions = Spree::PaymentSessions::PaypalCheckout.unscoped.where(payment_method_id: payment.payment_method_id, **owner)+ sessions.find { |session| capture_id_in(session.external_data) == capture_id }&.external_id+ end++ # The older flow only ever paid for orders.+ def legacy_order_id(payment, capture_id)+ return if payment.order_id.nil? || !legacy_orders_table?++ rows = legacy_order_model.where(payment_method_id: payment.payment_method_id, order_id: payment.order_id)+ rows.find { |row| capture_id_in(row.data) == capture_id }&.paypal_id+ end++ def capture_id_in(data)+ data = JSON.parse(data) if data.is_a?(String)+ return unless data.is_a?(Hash)++ data.dig('purchase_units', 0, 'payments', 'captures', 0, 'id')+ rescue JSON::ParserError+ nil+ end++ def legacy_orders_table?+ @legacy_orders_table = ActiveRecord::Base.connection.table_exists?(LEGACY_ORDERS_TABLE) if @legacy_orders_table.nil?+ @legacy_orders_table+ end++ # The 5.x model is gone, so the table is read through a throwaway one.+ def legacy_order_model+ @legacy_order_model ||= Class.new(ActiveRecord::Base) do+ self.table_name = LEGACY_ORDERS_TABLE+ self.inheritance_column = nil+ end+ end+ end+ end+enddiff --git a/spree/providers/paypal/lib/tasks/register_paypal_webhooks.rake b/spree/providers/paypal/lib/tasks/register_paypal_webhooks.rakenew file mode 100644index 0000000000..4630c5d463--- /dev/null+++ b/spree/providers/paypal/lib/tasks/register_paypal_webhooks.rake@@ -0,0 +1,36 @@+namespace :spree do+ namespace :upgrade do+ desc <<~DESC+ Registers the PayPal webhook of every PayPal payment method and stores+ its id on the method, replacing the events of a webhook created by hand+ under the spree_paypal_checkout 0.x guide.++ Run once after upgrading to Spree 6.0. Without it, PayPal methods+ created before the upgrade have no webhook id and reject every+ delivery.++ Safe to run again.+ DESC+ task register_paypal_webhooks: :environment do+ registered = 0+ failed = []++ SpreePaypalCheckout::Gateway.find_each do |payment_method|+ webhook = SpreePaypalCheckout::RegisterWebhook.new.call(payment_method: payment_method)+ if webhook+ registered += 1+ else+ failed << { id: payment_method.id, error: 'it has no webhook URL; its store has no URL' }+ end+ rescue StandardError => e+ failed << { id: payment_method.id, error: e.message }+ end++ puts "Registered the webhook of #{registered} PayPal payment method(s)."+ next if failed.empty?++ failed.each { |failure| puts " payment method #{failure[:id]}: #{failure[:error]}" }+ abort 'Some PayPal webhooks could not be registered. Re-run after resolving the errors above.'+ end+ end+enddiff --git a/spree/providers/paypal/spec/lib/tasks/register_paypal_webhooks_spec.rb b/spree/providers/paypal/spec/lib/tasks/register_paypal_webhooks_spec.rbnew file mode 100644index 0000000000..d2cc6e4835--- /dev/null+++ b/spree/providers/paypal/spec/lib/tasks/register_paypal_webhooks_spec.rb@@ -0,0 +1,43 @@+require 'spec_helper'+require 'rake'++Rake::Task.define_task(:environment) unless Rake::Task.task_defined?(:environment)+load SpreePaypalCheckout::Engine.root.join('lib', 'tasks', 'register_paypal_webhooks.rake')++RSpec.describe 'spree:upgrade:register_paypal_webhooks' do+ subject(:run_task) do+ task = Rake::Task['spree:upgrade:register_paypal_webhooks']+ task.reenable+ task.invoke+ end++ let(:store) { @default_store }+ let!(:first) { create(:paypal_checkout_gateway, store: store) }+ let!(:second) { create(:paypal_checkout_gateway, store: store) }+ let(:service) { instance_double(SpreePaypalCheckout::RegisterWebhook) }++ before { allow(SpreePaypalCheckout::RegisterWebhook).to receive(:new).and_return(service) }++ it 'registers the webhook of every PayPal method' do+ allow(service).to receive(:call).and_return({ 'id' => 'WH-1' })++ expect { run_task }.to output(/Registered the webhook of 2 PayPal payment method/).to_stdout+ expect(service).to have_received(:call).with(payment_method: first)+ expect(service).to have_received(:call).with(payment_method: second)+ end++ it 'ignores other payment methods' do+ create(:check_payment_method, store: store)+ allow(service).to receive(:call).and_return({ 'id' => 'WH-1' })++ expect { run_task }.to output(/2 PayPal payment method/).to_stdout+ end++ it 'carries on past a failure and then stops with the reasons' do+ allow(service).to receive(:call).with(payment_method: first).and_raise(StandardError, 'PayPal answered 401')+ allow(service).to receive(:call).with(payment_method: second).and_return({ 'id' => 'WH-2' })++ expect { run_task }.to raise_error(SystemExit).+ and output(/Registered the webhook of 1 .*payment method #{first.id}: PayPal answered 401/m).to_stdout+ end+enddiff --git a/spree/providers/paypal/spec/maintenance_tasks/spree_paypal_checkout/maintenance_tasks/migrate_payments_spec.rb b/spree/providers/paypal/spec/maintenance_tasks/spree_paypal_checkout/maintenance_tasks/migrate_payments_spec.rbnew file mode 100644index 0000000000..10d105489f--- /dev/null+++ b/spree/providers/paypal/spec/maintenance_tasks/spree_paypal_checkout/maintenance_tasks/migrate_payments_spec.rb@@ -0,0 +1,154 @@+require 'spec_helper'++RSpec.describe SpreePaypalCheckout::MaintenanceTasks::MigratePayments do+ let(:store) { @default_store }+ let(:gateway) { create(:paypal_checkout_gateway, store: store) }+ let(:order) { create(:order, store: store, total: 10) }++ # The table is gone from 6.0 schemas, so it is built in the shape the 0.x gem+ # left behind, once and outside the example's transaction, because MySQL+ # commits any open transaction when a table is created.+ before(:all) do+ ActiveRecord::Base.connection.create_table(described_class::LEGACY_ORDERS_TABLE, force: true) do |t|+ t.bigint :order_id, null: false+ t.bigint :payment_method_id, null: false+ t.string :paypal_id, null: false+ t.decimal :amount, precision: 10, scale: 2, null: false+ t.json :data+ t.timestamps+ end+ end++ after(:all) do+ ActiveRecord::Base.connection.drop_table described_class::LEGACY_ORDERS_TABLE, if_exists: true+ end++ def run_task(dry_run: false)+ result = Spree::MaintenanceTasks::Start.call(task_name: described_class.name, initiated_via: 'cli', dry_run: dry_run)+ Spree::MaintenanceTasks::RunJob.perform_now(result.value.id)+ result.value.reload+ end++ def capture_data(capture_id)+ { 'id' => 'IGNORED', 'status' => 'COMPLETED',+ 'purchase_units' => [{ 'payments' => { 'captures' => [{ 'id' => capture_id, 'status' => 'COMPLETED' }] } }] }+ end++ def payment_with_capture(capture_id)+ create(:payment, order: order, payment_method: gateway, amount: 10, response_code: capture_id, status: 'completed')+ end++ def legacy_order_model+ Class.new(ActiveRecord::Base) { self.table_name = 'spree_paypal_checkout_orders' }+ end++ context 'with a payment taken through a 0.x payment session' do+ let!(:payment) { payment_with_capture('CAPTURESESSION001') }++ before do+ create(:paypal_checkout_payment_session, owner: order, payment_method: gateway, amount: 10,+ status: 'completed', external_id: 'SESSIONORDER00001',+ external_data: capture_data('CAPTURESESSION001'))+ end++ it "takes the session's order id and keeps the capture id in metadata" do+ run = run_task++ payment.reload+ expect(payment.response_code).to eq('SESSIONORDER00001')+ expect(payment.metadata['paypal_capture_id']).to eq('CAPTURESESSION001')+ expect(run.tallies['converted']).to eq(1)+ end++ it 'links the session to its payment again' do+ run_task++ expect(Spree::PaymentSession.find_by(external_id: 'SESSIONORDER00001').payment).to eq(payment.reload)+ end+ end++ context 'with a payment from the older non-session flow' do+ let!(:payment) { payment_with_capture('CAPTURELEGACY0001') }++ before do+ legacy_order_model.create!(order_id: order.id, payment_method_id: gateway.id, paypal_id: 'LEGACYORDER000001',+ amount: 10, data: capture_data('CAPTURELEGACY0001'))+ end++ it 'matches it through spree_paypal_checkout_orders' do+ run_task++ expect(payment.reload.response_code).to eq('LEGACYORDER000001')+ expect(payment.metadata['paypal_capture_id']).to eq('CAPTURELEGACY0001')+ end++ it 'writes nothing in a dry run' do+ run = run_task(dry_run: true)++ expect(payment.reload.response_code).to eq('CAPTURELEGACY0001')+ expect(run.tallies['would_convert']).to eq(1)+ end++ it 'converts nothing on a second run' do+ run_task+ run = run_task++ expect(payment.reload.response_code).to eq('LEGACYORDER000001')+ expect(run.tallies['converted'].to_i).to eq(0)+ expect(run.tallies['already_converted']).to eq(1)+ end+ end++ context 'with a payment it cannot match' do+ let!(:payment) { payment_with_capture('CAPTUREORPHAN0001') }++ it 'leaves it as it was, counts it and logs its id' do+ allow(Rails.logger).to receive(:warn).and_call_original++ run = run_task++ expect(payment.reload.response_code).to eq('CAPTUREORPHAN0001')+ expect(payment.metadata).not_to have_key('paypal_capture_id')+ expect(run.tallies['unmatched']).to eq(1)+ expect(Rails.logger).to have_received(:warn).with(/payment #{payment.id}\z/)+ end+ end++ context 'when another payment of the order already carries the PayPal order' do+ let!(:payment) { payment_with_capture('CAPTUREDUPLICATE1') }++ before do+ create(:payment, order: order, payment_method: gateway, amount: 10, response_code: 'DUPLICATEORDER001').+ update_columns(metadata: { 'paypal_capture_id' => 'CAPTURE60000000002' })+ legacy_order_model.create!(order_id: order.id, payment_method_id: gateway.id, paypal_id: 'DUPLICATEORDER001',+ amount: 10, data: capture_data('CAPTUREDUPLICATE1'))+ allow_any_instance_of(Spree::Payment).to receive(:update_columns).and_raise(ActiveRecord::RecordNotUnique)+ end++ it 'leaves it as it was and carries on' do+ run = run_task++ expect(run.status).to eq('succeeded')+ expect(payment.reload.response_code).to eq('CAPTUREDUPLICATE1')+ expect(run.tallies['unmatched']).to eq(1)+ end+ end++ it 'leaves payments taken in 6.0 alone' do+ payment = create(:payment, order: order, payment_method: gateway, amount: 10, response_code: 'ORDER60000000001')+ payment.update_columns(metadata: { 'paypal_capture_id' => 'CAPTURE60000000001' })++ run = run_task++ expect(payment.reload.response_code).to eq('ORDER60000000001')+ expect(run.tallies['already_converted']).to eq(1)+ end++ it "leaves other gateways' payments alone" do+ other = create(:payment, order: order, amount: 10, response_code: 'ch_123')++ run_task++ expect(other.reload.response_code).to eq('ch_123')+ end+enddiff --git a/spree/providers/paypal/spec/registers_upgrade_steps_spec.rb b/spree/providers/paypal/spec/registers_upgrade_steps_spec.rbnew file mode 100644index 0000000000..bd33fe568a--- /dev/null+++ b/spree/providers/paypal/spec/registers_upgrade_steps_spec.rb@@ -0,0 +1,35 @@+require 'spec_helper'+require 'spree/upgrade'++RSpec.describe 'the PayPal upgrade steps' do+ # The manifest marks both steps optional because they ship here rather than+ # in core; an installation with the gem must therefore find them.+ it 'registers the payment migration' do+ expect(Spree::MaintenanceTask.find_registered('SpreePaypalCheckout::MaintenanceTasks::MigratePayments')).to be_present+ end++ # A web or worker process never loads the Rakefile, so a run started from+ # the dashboard has to find the task in this gem.+ it 'finds the webhook task in this gem outside the command line' do+ path = Spree::MaintenanceTask.new.rake_file_defining('spree:upgrade:register_paypal_webhooks')++ expect(path).to start_with(SpreePaypalCheckout::Engine.root.to_s)+ end++ it 'offers both steps where the gem is installed' do+ %w[migrate_paypal_payments register_paypal_webhooks].each do |id|+ step = Spree::Upgrade.find_step(id)++ expect(step).to be_present+ expect(step['optional']).to be(true)+ expect(Spree::Upgrade.available?(step)).to be(true), "#{id} should be available"+ end+ end++ it 'runs both after the Stripe webhook key step' do+ ids = Spree::Upgrade.manifests.find { |manifest| manifest['to'] == '6.0' }['steps'].map { |step| step['id'] }++ expect(ids.index('migrate_paypal_payments')).to eq(ids.index('migrate_stripe_webhook_keys') + 1)+ expect(ids.index('register_paypal_webhooks')).to eq(ids.index('migrate_paypal_payments') + 1)+ end+end