Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.
Ruby b2b-commercee-commerceecommerceecommerce-apiecommerce-frameworkecommerce-platformheadlessheadless-commerceheadless-ecommercemarketplacemulti-tenantmulti-vendormulti-vendor-ecommerceopen-sourcespree-commerce
Fail an API request that repeats a query (#14638)
Damian Legawiec0e5878c15cf3
diff --git a/CLAUDE.md b/CLAUDE.mdindex e422dc5356..635431adac 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -500,6 +500,8 @@ Spree::Dependencies.cart_add_item_service = 'Spree::Cart::AddItem'### Performance- Use `includes`/`preload` to avoid N+1 queries (`ar_lazy_preload` gem also active)+- **Every API request in the test suite is scanned for N+1 queries** (Prosopite in raise mode, configured in the dummy app's `test.rb`). A `Prosopite::NPlusOneQueriesError` prints the repeated statement and the serializer or model line that issued it. Fix it at that line: add the association to `collection_includes`, read a loaded association instead of running a scoped query, or replace the per-record lookup with one grouped query. Never answer it with `count` or `exists?` per record — that is the same N+1 in another shape; `size` on an already-loaded association is free, a `count` is not. No endpoint is exempt — never fix it by disabling the guard, skipping a spec, or wrapping the read in `Prosopite.pause`. An allow-list entry (`Prosopite.allow_stack_paths` in that `test.rb`) is only for a lookup that legitimately runs once per record being written (a uniqueness probe, a workflow run per requested row) and carries its reason next to it.+- A list endpoint spec must render at least two records, since a repeated query cannot show with one — every `GET #index` block includes `it_behaves_like 'a list without N+1 queries'` with a `let!` creating them.- Use `Rails.cache` for expensive operations; use `cache_key_with_version` for custom keys- Proper database indexingdiff --git a/docs/plans/decisions.md b/docs/plans/decisions.mdindex 254cb2977d..cecdb8dd96 100644--- a/docs/plans/decisions.md+++ b/docs/plans/decisions.md@@ -1150,6 +1150,78 @@ changes only affect future numbers; and code saving a numbered record withnormally assigns one is `before_validation`. Plan:`6.0-document-numbers.md`.+## 2026-09-07: N+1 tripwire — every API request in the suite is scanned for repeated queries++Lazy preloading on list collections already batches association reads, so+the N+1s Spree still ships are per-record model queries a serializer+reaches: a stock check, a price lookup, a `count`, a derived number. Rails'+strict loading cannot see those (it watches association loading, which the+lazy preloader satisfies first), so the net is Prosopite, which fingerprints+the SQL a request ran and reports a statement repeated from one call site.++This repository's dummy app hooks a scan around every v3 request and raises+on a report. The hook lives in the dummy app's `test.rb`, not in the shipped+controller, so a host application that bundles Prosopite for its own use is+never opted into Spree's guard. Statements issued while a record is+validated, saved, locked, destroyed or committed are not reported: writes+run their callbacks one record at a time by design (a `dependent: :destroy`+cascade, a uniqueness or number probe per created row), and the guard+exists for what a request reads. The few workflows that write one record+per requested row — a cart's lines, a split checkout's seller orders, a+payout per currency, a product payload's nested variants — are named in+the same allow list, each with its reason.++Two traps are worth naming, because both make the guard lie. Prosopite+fingerprints through pg_query, which cannot parse SQLite's `?` binds, so+SQLite borrows the text normalizer written for MySQL — and that normalizer+blanks anything double-quoted, which is a string literal in MySQL but an+identifier in SQLite. Left alone it erases every table name, so unrelated+queries share one fingerprint and a single record's distinct associations+report as an N+1; re-quoting identifiers as backticks first fixes it, and+it accounted for more than half of the first run's failures. The second+has no fix at the guard: two associations on one table — an order's billing and shipping address, a category's image and square image —+are indistinguishable from a repeat in the SQL alone. Telling them apart by+the row each lookup asks for was tried and abandoned, because a genuine+`belongs_to` N+1 also fetches a different row every time, so the same rule+hid the commonest N+1 there is. The guard therefore reports these, and the+fix is to preload the pair at the render site; a review of this change+caught the earlier attempt.++No endpoint is exempt. Turning it on, with the fingerprinting corrected,+left 63 API spec files with a repeated query; they were fixed in the same+change rather than parked on a list that logs instead of failing, because+a file on such a list hides every new repeat added to its endpoints too.++Products, categories and collections were then taken first, as the+highest-traffic surfaces. Every listing and detail endpoint across the+three is now clean; what remains on them is write-path work, where a query+per record created is inherent. The fixes worth naming are structural: a+category's product counts and its ancestor chain each collapse to one+query using the nested set's own `lft`/`rgt` bounds rather than a query per+node; the storefront's category filter sidebar costs one query instead of+two per child; counter-cache recounts across a bulk assignment are grouped+rather than `reset_counters` per record; and bulk tagging reads the tags it+needs once. A category's breadcrumb trail now preloads the images it+renders.++The shape behind most of them: a model method that reads a loaded+association when one happens to be loaded and runs a scoped query when it+is not, which on a list means a query per row. Prices, store credit,+publications, confirmation checks and the cart's delivery-address question+all had it. Cart writes were the worst case — price queries scaled at three+per line item, so a five-line cart cost fifteen — because a cart built one+item at a time has no relation to batch from; registering the resolved+variants as one preload context makes it flat. Two candidates were reverted+after the core suite caught them: `category_and_ancestors` is memoized, so+calling it from a callback poisons later reads, and the line-item finder+must re-query because callers add lines behind its back. Two conventions+follow. A repeated query is fixed at the line the report names — the association goes into+`collection_includes`, or the model reads a loaded association instead of+running a scoped query per record (`read_association_in_memory?`) — never+by widening the allow list. And a+list endpoint's spec renders at least two records, since a repeated+statement cannot show with one.+## 2026-08-13: Isolation tripwires — the guard under the store-scoping disciplineOpen-core isolation is the controller discipline (every lookup through adiff --git a/spree/api/app/controllers/concerns/spree/api/v3/cart_resolvable.rb b/spree/api/app/controllers/concerns/spree/api/v3/cart_resolvable.rbindex 6eb1e4d5f5..3d7d9c91d2 100644--- a/spree/api/app/controllers/concerns/spree/api/v3/cart_resolvable.rb+++ b/spree/api/app/controllers/concerns/spree/api/v3/cart_resolvable.rb@@ -50,8 +50,15 @@ module Spree# Render the cart as JSON using the cart serializer. Rendering never# mutates: an action that wants unbuyable lines swept calls# +sweep_unbuyable_lines!+ itself first.+ #+ # Serialized from a fresh, lazily preloading read, since lazy+ # preloading does not reach lines and fulfillments a write built. The+ # warnings exist only in memory, so they are carried over.def render_cart(status: :ok)- render json: Spree.api.cart_serializer.new(@cart, params: serializer_params).to_h, status: status+ cart = current_store.carts.where(id: @cart.id).preload_associations_lazily.take!+ cart.warnings = @cart.warnings++ render json: Spree.api.cart_serializer.new(cart, params: serializer_params).to_h, status: statusend# Drops lines the customer can no longer buy — a discontinued product,@@ -80,7 +87,13 @@ module Spree# storefront that knows nothing about groups still finds every order it# needs there.def render_order(status: :ok)- record = @cart.reload+ # Read back through a lazily preloading relation rather than+ # reloaded in place: a group's orders, and everything each of them+ # renders, then load once for all of them instead of once per order.+ purchases = @cart.is_a?(Spree::OrderGroup) ? current_store.order_groups : current_store.orders+ # Chained through `where` first: an association collection drops the+ # lazy flag when `find` rebuilds the query from the association.+ record = purchases.where(id: @cart.id).preload_associations_lazily.take!render json: purchase_serializer_for(record).new(record, params: serializer_params).to_h, status: statusenddiff --git a/spree/api/app/controllers/concerns/spree/api/v3/orders/fulfillment_actions.rb b/spree/api/app/controllers/concerns/spree/api/v3/orders/fulfillment_actions.rbindex 4f8f495bf4..cca9b8e73f 100644--- a/spree/api/app/controllers/concerns/spree/api/v3/orders/fulfillment_actions.rb+++ b/spree/api/app/controllers/concerns/spree/api/v3/orders/fulfillment_actions.rb@@ -72,8 +72,13 @@ module Spree)if changer.run!+ # Read back through a lazily preloading query: what each parcel+ # renders, its number included, then loads once for all of them.+ order = current_store.orders.where(id: @order.id).preload_associations_lazily.take!+ fulfillments = order.fulfillments.to_a+render json: {- data: @order.reload.fulfillments.map { |fulfillment| serialize_resource(fulfillment) }+ data: fulfillments.map { |fulfillment| serialize_resource(fulfillment) }}elserender_validation_error(changer.errors)diff --git a/spree/api/app/controllers/spree/api/v3/admin/customer_groups_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/customer_groups_controller.rbindex 42bd70fda5..ea2d2b67b4 100644--- a/spree/api/app/controllers/spree/api/v3/admin/customer_groups_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/customer_groups_controller.rb@@ -25,6 +25,10 @@ module SpreeSpree.api.admin_customer_group_serializerend+ def collection_includes+ [:customer_group_users]+ end+def permitted_paramsparams.permit(*permitted_attributes)enddiff --git a/spree/api/app/controllers/spree/api/v3/admin/customers_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/customers_controller.rbindex 9a9ff5e32e..86002ec42a 100644--- a/spree/api/app/controllers/spree/api/v3/admin/customers_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/customers_controller.rb@@ -179,12 +179,12 @@ module Spree# Bulk add the given customers to the given groups. Idempotent —# customers already in a group are skipped at the model layer.def bulk_add_to_groups- apply_groups(:add_customers)+ apply_groups(:add_customers_to)end# Bulk remove the given customers from the given groups.def bulk_remove_from_groups- apply_groups(:remove_customers)+ apply_groups(:remove_customers_from)endprotected@@ -295,8 +295,9 @@ module Spreeend# Authorises bulk group mutation, decodes prefixed IDs, then dispatches- # to `add_customers` / `remove_customers` per group. Returns the- # counts of records actually affected so the UI can show a toast.+ # to `add_customers_to` / `remove_customers_from` for all groups at+ # once. Returns the counts of records actually affected so the UI can+ # show a toast.def apply_groups(method)authorize! :update, model_class@@ -304,9 +305,9 @@ module Spreegroup_ids = decode_ids(params[:customer_group_ids])scoped_user_ids = scope.where(id: user_ids).pluck(:id)- scoped_groups = Spree::CustomerGroup.for_store(current_store).where(id: group_ids)+ scoped_groups = Spree::CustomerGroup.for_store(current_store).where(id: group_ids).to_a- scoped_groups.find_each { |group| group.public_send(method, scoped_user_ids) }+ Spree::CustomerGroup.public_send(method, scoped_groups, scoped_user_ids)render json: { customer_count: scoped_user_ids.size, customer_group_count: scoped_groups.size }enddiff --git a/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles/origin_groups_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles/origin_groups_controller.rbindex daf254c826..7064e932ce 100644--- a/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles/origin_groups_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles/origin_groups_controller.rb@@ -57,7 +57,7 @@ module Spreeenddef collection_includes- [:stock_locations]+ [:stock_locations, :delivery_zones, :delivery_methods]endprivatediff --git a/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles_controller.rbindex bc36b062e5..d8ee67f176 100644--- a/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/delivery_profiles_controller.rb@@ -74,7 +74,7 @@ module Spreeenddef collection_includes- [delivery_origin_groups: :stock_locations]+ [:delivery_methods, :delivery_zones, { delivery_origin_groups: :stock_locations }]endprivatediff --git a/spree/api/app/controllers/spree/api/v3/admin/me_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/me_controller.rbindex dab44bcc30..c02cc563c9 100644--- a/spree/api/app/controllers/spree/api/v3/admin/me_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/me_controller.rb@@ -103,6 +103,8 @@ module Spree# role would both disclose more than was handed over and mislead a# client that reads /me to decide what to attempt.def me_response+ ActiveRecord::Associations::Preloader.new(records: [current_user], associations: { spree_roles: :resource }).call+{user: admin_user_serializer.new(current_user, params: serializer_params).to_h,permissions: serialize_permissions(current_ability),diff --git a/spree/api/app/controllers/spree/api/v3/admin/option_types_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/option_types_controller.rbindex accc0cbee1..e3fc1c03ef 100644--- a/spree/api/app/controllers/spree/api/v3/admin/option_types_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/option_types_controller.rb@@ -42,6 +42,19 @@ module Spree[:option_values]end+ # Values created or changed by this write were never loaded with+ # their swatches, so fetch every value's image in one query before+ # rendering.+ def serialize_resource(resource)+ if expand_list.include?('option_values')+ ActiveRecord::Associations::Preloader.new(+ records: resource.option_values.to_a,+ associations: { image_attachment: :blob }+ ).call+ end+ super+ end+def permitted_paramsparams.permit(*permitted_attributes)enddiff --git a/spree/api/app/controllers/spree/api/v3/admin/orders_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/orders_controller.rbindex 60e7a2449f..634a414bfb 100644--- a/spree/api/app/controllers/spree/api/v3/admin/orders_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/orders_controller.rb@@ -233,7 +233,9 @@ module Spree# `fields` list names an order's attributes, and narrowing a group by# it can drop the very keys that say it is one.def serialize_completion(record)- return serialize_resource(record.reload) unless record.is_a?(Spree::OrderGroup)+ # Read back lazily preloading, as #show is, so its fulfillments and+ # their rows load once for all of them rather than per fulfillment.+ return serialize_resource(Spree::Order.preload_associations_lazily.find(record.id)) unless record.is_a?(Spree::OrderGroup)Spree.api.admin_order_group_serializer.new(completed_group(record), params: serializer_params).to_hend@@ -245,7 +247,7 @@ module Spree:customer, :ship_address, :bill_address,{ payments: %i[payment_method source] },{ orders: [:seller, :market, :gift_card, :customer, { line_items: { variant: :prices } }] }- ).find(group.id)+ ).preload_associations_lazily.find(group.id)end# Override scope — Order uses SingleStoreResource (for_store).@@ -300,8 +302,12 @@ module Spree# are loaded with their selected rate because the freight summary# reads it — otherwise a page of orders costs a query per# fulfillment for a field that is nil on every parcel order.+ # Both addresses are named because they are two associations on+ # one table, so a row that reads them separately costs a lookup+ # each.[:customer, :channel, :seller, :external_references, :cancel_reason,:market, { fulfillments: %i[selected_delivery_rate external_references] },+ :bill_address, :ship_address,{ line_items: [:external_references, { variant: :prices }, { price_list: :catalog }] },{ po_document_attachment: :blob }]enddiff --git a/spree/api/app/controllers/spree/api/v3/admin/price_lists_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/price_lists_controller.rbindex 6b18e6bfd1..5d284cfa78 100644--- a/spree/api/app/controllers/spree/api/v3/admin/price_lists_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/price_lists_controller.rb@@ -123,8 +123,26 @@ module Spree[:price_adjustment_tiers]end+ def serializer_params+ return super unless action_name == 'index'++ super.merge(page_counts)+ end+private+ # One grouped count per figure for the whole page rather than two+ # queries per list.+ def page_counts+ @page_counts ||= begin+ price_list_ids = collection.map(&:id)+ {+ products_counts: Spree::PriceList.products_count_by_list(price_list_ids),+ prices_counts: Spree::PriceList.prices_count_by_list(price_list_ids)+ }+ end+ end+# Loads the record without the action-derived authorization# `set_resource` runs (which would check `:activate` /# `:deactivate` — actions that abilities don't grant). Thediff --git a/spree/api/app/controllers/spree/api/v3/admin/products/variants_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/products/variants_controller.rbindex 5f74d8a999..f59e325e48 100644--- a/spree/api/app/controllers/spree/api/v3/admin/products/variants_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/products/variants_controller.rb@@ -28,8 +28,11 @@ module Spreeauthorize!(:show, @parent)end+ # `active_stock_reservations` is what lets the quantifier count+ # held stock in memory; without it the serializer's availability+ # column sums reservations once per variant.def scope_includes- [:prices, stock_levels: :stock_location]+ [:prices, { stock_levels: [:stock_location, :active_stock_reservations] }]enddef create_workflowdiff --git a/spree/api/app/controllers/spree/api/v3/admin/products_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/products_controller.rbindex 55ee2de35c..eab1ceaaa6 100644--- a/spree/api/app/controllers/spree/api/v3/admin/products_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/products_controller.rb@@ -219,7 +219,7 @@ module Spreechannels = scoped_channelsproduct_ids = bulk_collection.distinct.ids- channels.find_each { |channel| channel.add_products(product_ids) }+ channels.add_products(product_ids)render json: { product_count: product_ids.size, channel_count: channels.size }end@@ -231,7 +231,7 @@ module Spreechannels = scoped_channelsproduct_ids = bulk_collection.distinct.ids- removed = channels.sum { |channel| channel.remove_products(product_ids) }+ removed = channels.remove_products(product_ids)render json: { product_count: product_ids.size, channel_count: channels.size, removed: removed }end@@ -293,6 +293,14 @@ module Spree{ media: Spree::Media, variants: Spree::Variant }end+ # A written product is serialized from a fresh read: lazy preloading+ # does not reach records a write built, so the instance the workflow+ # returned would query once per variant.+ def serialize_resource(resource)+ resource = scope.where(id: resource.id).preload_associations_lazily.take! if resource.lazy_preload_context.nil?+ super+ end+# Use SearchProvider::Database for collection to handle price/best_selling# sorting correctly (counts before sorting, avoiding PG/Mobility issues).def collectiondiff --git a/spree/api/app/controllers/spree/api/v3/admin/sellers/payouts_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/sellers/payouts_controller.rbindex cab0e7a10f..f19a7752b8 100644--- a/spree/api/app/controllers/spree/api/v3/admin/sellers/payouts_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/sellers/payouts_controller.rb@@ -30,7 +30,8 @@ module Spree# because another went through: the payout row exists and is# failed or unresolved, and an operator reading "settled" would# not know to go looking for it.- body = { data: settled.map { |payout| serialize_payout(payout) } }+ transfer_counts = Spree::SellerTransfer.where(payout_id: settled.map(&:id)).group(:payout_id).count+ body = { data: settled.map { |payout| serialize_payout(payout, transfer_counts) } }body[:meta] = { failures: failures } if failures.any?render json: body, status: :created@@ -96,8 +97,8 @@ module Spree@seller.payable_currenciesend- def serialize_payout(payout)- Spree.api.admin_seller_payout_serializer.new(payout, params: { store: current_store }).to_h+ def serialize_payout(payout, transfer_counts)+ Spree.api.admin_seller_payout_serializer.new(payout, params: { store: current_store, transfer_counts: transfer_counts }).to_hendendenddiff --git a/spree/api/app/controllers/spree/api/v3/admin/stock_transfers_controller.rb b/spree/api/app/controllers/spree/api/v3/admin/stock_transfers_controller.rbindex c896b4b56d..3d3af9d8fa 100644--- a/spree/api/app/controllers/spree/api/v3/admin/stock_transfers_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/admin/stock_transfers_controller.rb@@ -202,9 +202,10 @@ module Spreeend# The listing and the detail page both read a transfer's totals off- # its lines.+ # its lines. Both warehouses come from one table, so their external+ # references are loaded here rather than once per warehouse.def scope_includes- [line_includes]+ [{ source_location: :external_references, destination_location: :external_references }, line_includes]endendenddiff --git a/spree/api/app/controllers/spree/api/v3/resource_controller.rb b/spree/api/app/controllers/spree/api/v3/resource_controller.rbindex f4935a18bd..6760fcde73 100644--- a/spree/api/app/controllers/spree/api/v3/resource_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/resource_controller.rb@@ -248,8 +248,11 @@ module Spreeend# Finds a single resource within scope using prefixed ID+ # Loaded through the lazy-preload relation, as the collection is, so+ # the serializer's walk over the record's children batches their+ # associations instead of querying once per child.def find_resource- scope.find_by_prefix_id!(params[:id])+ scope.preload_associations_lazily.find_by_prefix_id!(params[:id])end# Per-record authorization hook. A no-op in the shared base: the Storediff --git a/spree/api/app/controllers/spree/api/v3/seller/orders_controller.rb b/spree/api/app/controllers/spree/api/v3/seller/orders_controller.rbindex 01c84ba7bd..23cfe0ece5 100644--- a/spree/api/app/controllers/spree/api/v3/seller/orders_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/seller/orders_controller.rb@@ -84,7 +84,7 @@ module Spree)if result.success?- render json: serialize_resource(@resource.reload)+ render json: serialize_resource(reloaded_resource)elserender_service_error(@resource.errors.presence || result.error)end@@ -115,7 +115,7 @@ module Spreenext render_validation_error(invalid.errors) if invalidif @resource.update(addresses)- render json: serialize_resource(@resource.reload)+ render json: serialize_resource(reloaded_resource)elserender_validation_error(@resource.errors)end@@ -143,12 +143,20 @@ module Spreesuper.not_draftsend+ # Both addresses are named because they are two associations on one+ # table, so a row that reads them separately costs a lookup each.def collection_includes- [:line_items, :customer, :channel]+ [:line_items, :customer, :channel, :bill_address, :ship_address]endprivate+ # Read back as #show reads it, lazily preloading, so its+ # fulfillments' rates and methods load once rather than per row.+ def reloaded_resource+ scope.preload_associations_lazily.find(@resource.id)+ end+# Either address, as a correction rather than a replacement: the# sent fields are laid over what the order already holds, so a# request naming one line does not blank out the country anddiff --git a/spree/api/app/controllers/spree/api/v3/seller/products_controller.rb b/spree/api/app/controllers/spree/api/v3/seller/products_controller.rbindex 56a26692d1..fc822c5c1f 100644--- a/spree/api/app/controllers/spree/api/v3/seller/products_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/seller/products_controller.rb@@ -142,8 +142,10 @@ module Spreesubmitted_by: try_spree_current_user)# A product Propose refused is untouched; one it accepted is in- # review or already through it.- product.reload.status != was+ # review or already through it. The saved value, not the+ # attribute: a refused update leaves the new status unsaved on+ # the record, and a rolled-back one restores what was stored.+ product.status_in_database != was}enddiff --git a/spree/api/app/controllers/spree/api/v3/seller/team_controller.rb b/spree/api/app/controllers/spree/api/v3/seller/team_controller.rbindex 5f71777a2f..a7c9c65e92 100644--- a/spree/api/app/controllers/spree/api/v3/seller/team_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/seller/team_controller.rb@@ -86,7 +86,7 @@ module Spreeprivatedef members- @members ||= current_seller.users.to_a+ @members ||= current_seller.users.with_attached_avatar.to_aenddef set_memberdiff --git a/spree/api/app/controllers/spree/api/v3/store/customers_controller.rb b/spree/api/app/controllers/spree/api/v3/store/customers_controller.rbindex 3cfac0fc2d..2b3ae2fc6a 100644--- a/spree/api/app/controllers/spree/api/v3/store/customers_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/store/customers_controller.rb@@ -76,6 +76,7 @@ module Spree# GET /api/v3/store/customerdef show+ preload_default_addressesrender json: serialize_resource(current_user)end@@ -120,6 +121,12 @@ module Spreeprivate+ # Both default addresses live in one table, so they are fetched in one+ # query rather than one each.+ def preload_default_addresses+ ActiveRecord::Associations::Preloader.new(records: [current_user], associations: %i[bill_address ship_address]).call+ end+def sensitive_update?(params[:email].present? && params[:email] != current_user.email) ||params[:password].present?diff --git a/spree/api/app/controllers/spree/api/v3/store/delivery_methods_controller.rb b/spree/api/app/controllers/spree/api/v3/store/delivery_methods_controller.rbindex 61e6d61230..88146148b5 100644--- a/spree/api/app/controllers/spree/api/v3/store/delivery_methods_controller.rb+++ b/spree/api/app/controllers/spree/api/v3/store/delivery_methods_controller.rb@@ -93,7 +93,7 @@ module Spreedef covers_cart?(location, cart)return true if location.pickup_stock_policy == 'any'- cart.line_items.includes(variant: :product).all? do |line_item|+ pickup_line_items(cart).all? do |line_item|next true if line_item.variant.digital?next true unless line_item.variant.should_track_inventory?@@ -101,6 +101,11 @@ module Spreestock_level.present? && stock_level.available_count >= line_item.quantityendend++ # Loaded once for every location the cart is checked against.+ def pickup_line_items(cart)+ @pickup_line_items ||= cart.line_items.includes(variant: [:delivery_profile, { product: %i[delivery_profile store] }]).to_a+ endendendenddiff --git a/spree/api/app/serializers/spree/api/v3/admin/delivery_origin_group_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/delivery_origin_group_serializer.rbindex 60e77fa7a0..c0dbf8b65a 100644--- a/spree/api/app/serializers/spree/api/v3/admin/delivery_origin_group_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/delivery_origin_group_serializer.rb@@ -19,11 +19,11 @@ module Spreeendattribute :delivery_zones_count do |record|- record.delivery_zones.count+ record.delivery_zones.sizeendattribute :delivery_methods_count do |record|- record.delivery_methods.count+ record.delivery_methods.sizeendendenddiff --git a/spree/api/app/serializers/spree/api/v3/admin/delivery_profile_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/delivery_profile_serializer.rbindex 160a1f1ca7..f50723b9ad 100644--- a/spree/api/app/serializers/spree/api/v3/admin/delivery_profile_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/delivery_profile_serializer.rb@@ -47,16 +47,14 @@ module Spreeendend- attribute :products_count do |record|- record.products.count- end+ attribute :products_count, &:products_countattribute :delivery_methods_count do |record|- record.delivery_methods.count+ record.delivery_methods.sizeendattribute :delivery_zones_count do |record|- record.delivery_zones.count+ record.delivery_zones.sizeendendenddiff --git a/spree/api/app/serializers/spree/api/v3/admin/price_history_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/price_history_serializer.rbindex d0627e59d6..bd283a66a2 100644--- a/spree/api/app/serializers/spree/api/v3/admin/price_history_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/price_history_serializer.rb@@ -13,7 +13,7 @@ module Spreecreated_at: :stringattribute :variant_id do |price_history|- price_history.variant.prefixed_id+ Spree::Variant.prefixed_id_for(price_history.variant_id)endattribute :price_id do |price_history|diff --git a/spree/api/app/serializers/spree/api/v3/admin/price_list_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/price_list_serializer.rbindex 1ea28aebb2..86f74d3f4c 100644--- a/spree/api/app/serializers/spree/api/v3/admin/price_list_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/price_list_serializer.rb@@ -34,13 +34,18 @@ module Spreeattribute :currently_active, &:currently_active?# Cheap counts so the index can render "12 products / 36 prices"- # without forcing each row to expand the children.+ # without forcing each row to expand the children. The index counts+ # the whole page at once; a single list asks for its own.attribute :products_count do |pl|- pl.products.count+ counts = params[:products_counts]++ (counts || Spree::PriceList.products_count_by_list(pl.id)).fetch(pl.id, 0)endattribute :prices_count do |pl|- pl.prices.count+ counts = params[:prices_counts]++ counts ? counts.fetch(pl.id, 0) : pl.prices.countend# Always expanded: a list's bands are the rest of the sentence itsdiff --git a/spree/api/app/serializers/spree/api/v3/admin/price_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/price_serializer.rbindex 4bfbf3953f..2a4ec63b2b 100644--- a/spree/api/app/serializers/spree/api/v3/admin/price_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/price_serializer.rb@@ -17,7 +17,7 @@ module Spreeattributes :min_quantityattribute :variant_id do |price|- price&.variant&.prefixed_id+ Spree::Variant.prefixed_id_for(price&.variant_id)endattributes created_at: :iso8601, updated_at: :iso8601diff --git a/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rbindex 3c26066c5c..409972c8e6 100644--- a/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/seller_payout_serializer.rb@@ -32,8 +32,8 @@ module Spree## Read from a count the controller made for the whole page when there# is one; a single payout read falls back to asking for its own.- attribute(:transfers_count) do |payout, params|- counts = params && params[:transfer_counts]+ attribute(:transfers_count) do |payout|+ counts = params[:transfer_counts]counts ? counts.fetch(payout.id, 0) : payout.transfers.countenddiff --git a/spree/api/app/serializers/spree/api/v3/admin/seller_serializer.rb b/spree/api/app/serializers/spree/api/v3/admin/seller_serializer.rbindex 172b2845d9..df6f68f188 100644--- a/spree/api/app/serializers/spree/api/v3/admin/seller_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/admin/seller_serializer.rb@@ -54,9 +54,7 @@ module Spree# so the minimum-products requirement asks the same question free.attributes :products_count- attribute :users_count do |seller|- seller.users.size- end+ attributes :users_count# How far through the marketplace's checklist this seller is — what# the list's progress column and the profile's badge render. Thediff --git a/spree/api/app/serializers/spree/api/v3/seller/payout_serializer.rb b/spree/api/app/serializers/spree/api/v3/seller/payout_serializer.rbindex 7ac000b9ad..9d905fea62 100644--- a/spree/api/app/serializers/spree/api/v3/seller/payout_serializer.rb+++ b/spree/api/app/serializers/spree/api/v3/seller/payout_serializer.rb@@ -22,8 +22,8 @@ module Spree# How many earnings this settlement covers. Read from a count the# controller made for the whole page when there is one; a single# payout read falls back to asking for its own.- attribute(:transfers_count) do |payout, params|- counts = params && params[:transfer_counts]+ attribute(:transfers_count) do |payout|+ counts = params[:transfer_counts]counts ? counts.fetch(payout.id, 0) : payout.transfers.countenddiff --git a/spree/api/app/services/spree/api/v3/filters_aggregator.rb b/spree/api/app/services/spree/api/v3/filters_aggregator.rbindex 4a3e92e13d..a3dbcd25dd 100644--- a/spree/api/app/services/spree/api/v3/filters_aggregator.rb+++ b/spree/api/app/services/spree/api/v3/filters_aggregator.rb@@ -251,30 +251,75 @@ module Spreedef category_filterreturn nil if @category.nil?- # Get child categories at the next depth level- child_categories = @category.children.order(:lft).select do |child|- # Only include categories that have products in the current scope- @scope.in_category(child).exists?+ children = @category.children.order(:lft).to_a+ return nil if children.empty?++ counts = product_counts_by_child_category(children)+ # A child with nothing in the current scope is not offered as a+ # filter, which is what the per-child `exists?` decided before.+ options = children.filter_map do |child|+ count = counts[child.id]+ next if count.nil? || count.zero?++ {+ id: child.prefixed_id,+ name: child.name,+ permalink: child.permalink,+ count: count+ }end- return nil if child_categories.empty?+ return nil if options.empty?- {- id: 'categories',- type: 'category',- options: child_categories.map { |c| category_option_data(c) }- }+ { id: 'categories', type: 'category', options: options }end- def category_option_data(category)- count = @scope.in_category(category).distinct.count+ # Distinct product counts for each child category, counting its whole+ # subtree, in one query rather than two per child (an existence probe+ # and a count). A category sidebar is drawn on every filtered+ # storefront request, so the per-child pair is paid on each one.+ #+ # @param children [Array<Spree::Category>]+ # @return [Hash{Integer => Integer}] child category id => product count+ def product_counts_by_child_category(children)+ descendant_ids_by_child = descendant_ids_for(children)+ all_descendant_ids = descendant_ids_by_child.values.flatten.uniq+ return {} if all_descendant_ids.empty?++ product_categories = Spree::ProductCategory.table_name+ # product id => the categories it sits in, narrowed to this subtree+ pairs = @scope.reorder('').joins(:product_categories).+ where("#{product_categories}.category_id" => all_descendant_ids).+ distinct.+ pluck("#{product_categories}.category_id", "#{product_categories}.product_id")++ product_ids_by_category = Hash.new { |hash, key| hash[key] = [] }+ pairs.each { |category_id, product_id| product_ids_by_category[category_id] << product_id }++ descendant_ids_by_child.transform_values do |descendant_ids|+ descendant_ids.flat_map { |id| product_ids_by_category[id] }.uniq.size+ end+ end- {- id: category.prefixed_id,- name: category.name,- permalink: category.permalink,- count: count- }+ # Every child's subtree ids in one query. A nested set stores a+ # subtree as an `lft`/`rgt` range, so one read of the categories+ # spanned by the children answers for all of them — asking each child+ # separately is a query per filter option.+ #+ # @param children [Array<Spree::Category>]+ # @return [Hash{Integer => Array<Integer>}] child id => subtree ids+ def descendant_ids_for(children)+ bounds = children.to_h { |child| [child.id, (child.lft..child.rgt)] }++ rows = Spree::Category.+ where(store_id: children.first.store_id).+ where(lft: children.map(&:lft).min..children.map(&:rgt).max).+ pluck(:id, :lft)++ children.to_h do |child|+ range = bounds[child.id]+ [child.id, rows.filter_map { |id, lft| id if range.cover?(lft) }]+ endendendenddiff --git a/spree/api/lib/spree/api/testing_support/v3/base.rb b/spree/api/lib/spree/api/testing_support/v3/base.rbindex 3bd0b6e0ea..0eb6aa76b5 100644--- a/spree/api/lib/spree/api/testing_support/v3/base.rb+++ b/spree/api/lib/spree/api/testing_support/v3/base.rb@@ -178,6 +178,24 @@ shared_examples 'returns 422 Unprocessable Entity' doendend+# Prosopite reports a statement only once it repeats, and most specs create a+# single record, so a list endpoint's N+1 stays invisible unless the spec+# renders several rows. Include this in every `GET #index` block with a+# `subject` that performs the request and a `let!` creating at least two+# records the serializer will render:+#+# it_behaves_like 'a list without N+1 queries' do+# let!(:records) { create_list(:product, 3, store: store) }+# end+shared_examples 'a list without N+1 queries' do+ it 'renders several records without repeating a query' do+ subject++ expect(response).to have_http_status(:ok)+ expect(json_response['data'].size).to be >= 2+ end+end+shared_examples 'requires API key' docontext 'without API key' dolet(:headers) { {} }diff --git a/spree/api/spec/controllers/spree/api/v3/admin/payment_methods_controller_spec.rb b/spree/api/spec/controllers/spree/api/v3/admin/payment_methods_controller_spec.rbindex 08a8662bc2..d0d96a5769 100644--- a/spree/api/spec/controllers/spree/api/v3/admin/payment_methods_controller_spec.rb+++ b/spree/api/spec/controllers/spree/api/v3/admin/payment_methods_controller_spec.rb@@ -10,6 +10,12 @@ RSpec.describe Spree::Api::V3::Admin::PaymentMethodsController, type: :controllebefore { request.headers.merge!(headers) }describe 'GET #index' do+ it_behaves_like 'a list without N+1 queries' do+ subject { get :index, as: :json }++ let!(:second_payment_method) { create(:check_payment_method, store: store, name: 'Second method') }+ end+it 'returns store-scoped payment methods' doget :index, as: :jsondiff --git a/spree/api/spec/controllers/spree/api/v3/admin/price_lists_controller_spec.rb b/spree/api/spec/controllers/spree/api/v3/admin/price_lists_controller_spec.rbindex 7737e87ffa..4cf17bab79 100644--- a/spree/api/spec/controllers/spree/api/v3/admin/price_lists_controller_spec.rb+++ b/spree/api/spec/controllers/spree/api/v3/admin/price_lists_controller_spec.rb@@ -107,6 +107,19 @@ RSpec.describe Spree::Api::V3::Admin::PriceListsController, type: :controller doexpect(json_response['data'].pluck('name')).to eq(['Third list', 'Wholesale', 'Second list'])expect(json_response['data'].pluck('position')).to eq([1, 2, 3])end++ it 'counts a product priced on several variants once' do+ product = create(:product, store: store)+ [product.default_variant, create(:variant, product: product)].each do |variant|+ create(:price, variant: variant, price_list: price_list, currency: 'USD', amount: 10)+ end++ get :index, as: :json+ expect(json_response['data'].first).to include('products_count' => 1, 'prices_count' => 2)++ get :show, params: { id: price_list.prefixed_id }, as: :json+ expect(json_response).to include('products_count' => 1, 'prices_count' => 2)+ endenddescribe 'POST #create — one-shot creation' dodiff --git a/spree/api/spec/controllers/spree/api/v3/store/products_controller_spec.rb b/spree/api/spec/controllers/spree/api/v3/store/products_controller_spec.rbindex e27aa62a98..d11aa51fee 100644--- a/spree/api/spec/controllers/spree/api/v3/store/products_controller_spec.rb+++ b/spree/api/spec/controllers/spree/api/v3/store/products_controller_spec.rb@@ -95,6 +95,10 @@ RSpec.describe Spree::Api::V3::Store::ProductsController, type: :controller doenddescribe 'GET #index' do+ it_behaves_like 'a list without N+1 queries' do+ subject { get :index }+ end+it 'returns a list of products' doget :indexdiff --git a/spree/api/spec/serializers/spree/api/v3/product_serializer_spec.rb b/spree/api/spec/serializers/spree/api/v3/product_serializer_spec.rbindex 9b419b0964..edad4470dc 100644--- a/spree/api/spec/serializers/spree/api/v3/product_serializer_spec.rb+++ b/spree/api/spec/serializers/spree/api/v3/product_serializer_spec.rb@@ -85,7 +85,10 @@ RSpec.describe Spree::Api::V3::ProductSerializer do# +Product.available_on+ column. 6.0 drops the column fallback.describe 'available_on' dolet(:channel) { store.default_channel }- let(:publication) { product.product_publications.find_by(channel: channel) }+ # Read through the association the serializer itself uses, so a write+ # here is visible to it: `update_columns` writes straight to the row,+ # which a separately fetched copy would not carry back.+ let(:publication) { product.product_publications.find { |pub| pub.channel_id == channel.id } }before { Spree::Current.channel = channel }after { Spree::Current.channel = nil }diff --git a/spree/common_spree_dependencies.rb b/spree/common_spree_dependencies.rbindex dc7d1f3e3e..2fca243e38 100644--- a/spree/common_spree_dependencies.rb+++ b/spree/common_spree_dependencies.rb@@ -39,6 +39,14 @@ group :test dogem 'webmock'gem 'timecop'gem 'rails-controller-testing'+ # N+1 tripwire for API requests (see Spree::Api::V3::BaseController).+ # pg_query is Prosopite's SQL fingerprinter for every adapter but MySQL.+ # It has a C extension and no JRuby build, so it is scoped like the+ # database adapters above; the guard is inert where it is missing.+ gem 'prosopite'+ platforms :ruby do+ gem 'pg_query'+ endendgroup :test, :development dodiff --git a/spree/core/app/models/concerns/spree/derived_number.rb b/spree/core/app/models/concerns/spree/derived_number.rbindex 17f995d4fb..9c3e87972a 100644--- a/spree/core/app/models/concerns/spree/derived_number.rb+++ b/spree/core/app/models/concerns/spree/derived_number.rb@@ -58,7 +58,7 @@ module Spreeenddef sibling_association_name- self.class.name.demodulize.underscore.pluralize+ self.class.name.demodulize.underscore.pluralize.to_symendendenddiff --git a/spree/core/app/models/concerns/spree/has_external_references.rb b/spree/core/app/models/concerns/spree/has_external_references.rbindex 754051e089..eff4b13817 100644--- a/spree/core/app/models/concerns/spree/has_external_references.rb+++ b/spree/core/app/models/concerns/spree/has_external_references.rb@@ -43,8 +43,31 @@ module Spree# @return [Spree::ExternalReference, nil] nil when the reference was removeddef set_external_id(system, external_id, metadata: nil)key = Spree::ExternalReference.normalize_system(system)- reference = external_references.for_system(key).first+ write_external_reference(key, external_references.for_system(key).first, external_id, metadata)+ end+ # Writes several references at once, leaving systems the payload does not+ # mention untouched — a connector states its own key without disturbing+ # another system's. Accepts the `{ system => external_id }` map the admin+ # serializers render, or a list of `{ system:, external_id: }` entries.+ #+ # @param references [Hash{String,Symbol=>String}, Array<Hash>]+ # @return [void]+ def assign_external_references(references)+ entries = Spree::ExternalReference.normalize_references(references)+ return if entries.empty?++ keys = entries.map { |entry| Spree::ExternalReference.normalize_system(entry[:system]) }+ existing = external_references.where(system: keys).order(:id).group_by(&:system).transform_values(&:first)++ entries.zip(keys).each do |entry, key|+ existing[key] = write_external_reference(key, existing[key], entry[:external_id], entry[:metadata])+ end+ end++ private++ def write_external_reference(key, reference, external_id, metadata)if external_id.to_s.strip.blank?reference&.destroyexternal_references.reset@@ -64,21 +87,6 @@ module Spreereferenceend- # Writes several references at once, leaving systems the payload does not- # mention untouched — a connector states its own key without disturbing- # another system's. Accepts the `{ system => external_id }` map the admin- # serializers render, or a list of `{ system:, external_id: }` entries.- #- # @param references [Hash{String,Symbol=>String}, Array<Hash>]- # @return [void]- def assign_external_references(references)- Spree::ExternalReference.normalize_references(references).each do |reference|- set_external_id(reference[:system], reference[:external_id], metadata: reference[:metadata])- end- end-- private-# A reference belongs to the same store as the record it identifies.## Store-owning models answer directly;diff --git a/spree/core/app/models/concerns/spree/prefixed_id.rb b/spree/core/app/models/concerns/spree/prefixed_id.rbindex 994feb2d61..94fd2c5dc6 100644--- a/spree/core/app/models/concerns/spree/prefixed_id.rb+++ b/spree/core/app/models/concerns/spree/prefixed_id.rb@@ -123,12 +123,22 @@ module Spreereturn values unless klass+ # A model with a lookup of its own (Order) resolves each id through it.+ unless klass.method(:find_by_param).owner == Spree::PrefixedId::ClassMethods+ return values.map { |v| Spree::PrefixedId.prefixed_id?(v) ? klass.find_by_param!(v).id : v }+ end++ # One lookup for the whole list rather than one per id.+ decoded = values.select { |v| Spree::PrefixedId.prefixed_id?(v) }.index_with { |v| klass.decode_own_prefixed_id(v) }+ existing_ids = decoded.empty? ? [] : klass.where(id: decoded.values.compact).pluck(:id)+values.map do |v|- if Spree::PrefixedId.prefixed_id?(v)- klass.find_by_param!(v).id- else- v- end+ next v unless decoded.key?(v)++ id = decoded[v]+ raise ActiveRecord::RecordNotFound.new("Couldn't find #{klass.name} with param=#{v}", klass.name) unless id && existing_ids.include?(id)++ idendenddiff --git a/spree/core/app/models/concerns/spree/purchase/addresses.rb b/spree/core/app/models/concerns/spree/purchase/addresses.rbindex 1300090c26..9406391005 100644--- a/spree/core/app/models/concerns/spree/purchase/addresses.rb+++ b/spree/core/app/models/concerns/spree/purchase/addresses.rb@@ -62,7 +62,11 @@ module Spree## @return [Boolean]def shipping_address_required?- selected_methods = fulfillments.filter_map(&:delivery_method)+ candidates = fulfillments.to_a+ if candidates.many? && candidates.all?(ActiveRecord::Base)+ ActiveRecord::Associations::Preloader.new(records: candidates, associations: { selected_delivery_rate: :delivery_method }).call+ end+ selected_methods = candidates.filter_map(&:delivery_method)return selected_methods.any?(&:requires_address?) if selected_methods.any?return false if preferred_stock_location_id.present?@@ -270,9 +274,20 @@ module Spreecache[profile] = profile.present? && profile.requires_shipping_address?end- # Both profile paths are preloaded: the variant's own override, and the- # product's for the variants that inherit it.- line_items.includes(variant: [:delivery_profile, { product: :delivery_profile }]).any? do |line_item|+ # On a list the association batches across the page, so a fresh+ # `includes` query would load every cart's items separately. Read+ # alone, both profile paths are preloaded: the variant's own override,+ # and the product's for the variants that inherit it.+ profile_paths = { variant: [:delivery_profile, { product: :delivery_profile }] }+ items = if !read_association_in_memory?(:line_items)+ line_items.includes(profile_paths)+ elsif lazy_preload_context.nil?+ line_items.tap { |loaded| ActiveRecord::Associations::Preloader.new(records: loaded.to_a, associations: profile_paths).call }+ else+ line_items+ end++ items.any? do |line_item|profile_requires_address[line_item.variant&.resolved_delivery_profile]endenddiff --git a/spree/core/app/models/concerns/spree/purchase/digital_items.rb b/spree/core/app/models/concerns/spree/purchase/digital_items.rbindex c88eafd567..5909a4f28c 100644--- a/spree/core/app/models/concerns/spree/purchase/digital_items.rb+++ b/spree/core/app/models/concerns/spree/purchase/digital_items.rb@@ -9,7 +9,7 @@ module Spreeif total_quantity.zero? || line_items.empty?falseelse- line_items.includes(variant: :product).all?(&:digital?)+ line_items_with_variants(variant: :product).all?(&:digital?)endend@@ -18,7 +18,7 @@ module Spreeif total_quantity.zero? || line_items.empty?falseelse- line_items.includes(variant: :product).any?(&:digital?)+ line_items_with_variants(variant: :product).any?(&:digital?)endend@@ -27,7 +27,7 @@ module Spreeif total_quantity.zero? || line_items.empty?falseelse- line_items.includes(:variant).any?(&:with_digital_assets?)+ line_items_with_variants(variant: :digital_assets).any?(&:with_digital_assets?)endend@@ -55,6 +55,20 @@ module Spreedef delivery_step_required?line_items.any? && !digital?end++ private++ # Lines already in memory get the associations preloaded onto them —+ # onto every line of their preload context when they sit in a list, so+ # the list batches them instead of querying once per purchase.+ def line_items_with_variants(associations)+ return line_items.includes(associations) unless read_association_in_memory?(:line_items)++ loaded = line_items.to_a+ records = loaded.first&.lazy_preload_context&.records || loaded+ ActiveRecord::Associations::Preloader.new(records: records, associations: associations).call+ loaded+ endendendenddiff --git a/spree/core/app/models/concerns/spree/purchase/freight.rb b/spree/core/app/models/concerns/spree/purchase/freight.rbindex 6c92cb1c26..df19f87420 100644--- a/spree/core/app/models/concerns/spree/purchase/freight.rb+++ b/spree/core/app/models/concerns/spree/purchase/freight.rb@@ -42,7 +42,8 @@ module Spree# twice. Two freight consignments are still one load to the forwarder,# so they are merged rather than listed.def build_freight_summary- summaries = fulfillments.valid.filter_map { |fulfillment| fulfillment.selected_delivery_rate&.freight_summary }+ shipping = read_association_in_memory?(:fulfillments) ? fulfillments.reject(&:canceled?) : fulfillments.valid+ summaries = shipping.filter_map { |fulfillment| fulfillment.selected_delivery_rate&.freight_summary }return if summaries.empty?Spree::FreightSummary.merge(summaries)diff --git a/spree/core/app/models/concerns/spree/purchase/payment_processing.rb b/spree/core/app/models/concerns/spree/purchase/payment_processing.rbindex d2d098ca84..8ade4ac66d 100644--- a/spree/core/app/models/concerns/spree/purchase/payment_processing.rb+++ b/spree/core/app/models/concerns/spree/purchase/payment_processing.rb@@ -55,7 +55,7 @@ module Spree# @return [Boolean]def confirmation_required?Spree::StorePreferences.read(store, :always_include_confirm_step) ||- payments.valid.map(&:payment_method).compact.any?(&:confirmation_required?)+ valid_payments.filter_map(&:payment_method).any?(&:confirmation_required?)enddef unprocessed_payments@@ -64,6 +64,12 @@ module Spreeprivate+ # On a list the association batches across the page, so the loaded rows+ # are filtered rather than queried through `valid` once per record.+ def valid_payments+ read_association_in_memory?(:payments) ? payments.reject(&:has_invalid_status?) : payments.valid+ end+def process_payments_with(action)# Don't run if there is nothing to pay. Read from the database, not# the in-memory attribute: settlements land through the order statusdiff --git a/spree/core/app/models/concerns/spree/purchase/quantity_rules.rb b/spree/core/app/models/concerns/spree/purchase/quantity_rules.rbindex 302c6aa6cd..5277d8583b 100644--- a/spree/core/app/models/concerns/spree/purchase/quantity_rules.rb+++ b/spree/core/app/models/concerns/spree/purchase/quantity_rules.rb@@ -104,6 +104,8 @@ module Spree# accuse a completed sale of breaking a rule written after it.return [] if staff_initiated? || (is_a?(Spree::Order) && completed?)+ ActiveRecord::Associations::Preloader.new(records: line_items.to_a, associations: { variant: :product }).call+line_items.filter_map do |line_item|variant = line_item.variantnext if variant.nil?diff --git a/spree/core/app/models/concerns/spree/translatable_resource.rb b/spree/core/app/models/concerns/spree/translatable_resource.rbindex 2db0a1489d..4b5569608d 100644--- a/spree/core/app/models/concerns/spree/translatable_resource.rb+++ b/spree/core/app/models/concerns/spree/translatable_resource.rb@@ -168,7 +168,12 @@ module Spreedef translatable_storereturn self if is_a?(Spree::Store)- try(:store) || Spree::Current.store+ current_store = Spree::Current.store+ # The request's store instance already carries its markets, so a batch+ # of records does not re-read them once per record's own store copy.+ return current_store if current_store && try(:store_id) == current_store.id++ try(:store) || current_storeendprivatediff --git a/spree/core/app/models/spree/base.rb b/spree/core/app/models/spree/base.rbindex ee60fb7e3f..b36aac6757 100644--- a/spree/core/app/models/spree/base.rb+++ b/spree/core/app/models/spree/base.rb@@ -89,6 +89,30 @@ class Spree::Base < ApplicationRecordassociation(name).loaded? || lazy_preload_context.present?end+ # Recomputes a counter cache for many records at once. Rails'+ # +reset_counters+ spends two queries per record, which a bulk write pays+ # for every row it touched; this is one grouped count plus one UPDATE per+ # distinct value.+ #+ # @param ids [Array] the records whose counter is recomputed+ # @param association [Symbol] the has_many whose rows are counted+ # @return [void]+ def self.reset_counters_in_bulk(ids, association)+ ids = Array(ids).compact.uniq+ return if ids.empty?++ reflection = reflect_on_association(association)+ counter = reflection.klass.reflect_on_all_associations(:belongs_to).find do |belongs_to|+ belongs_to.foreign_key.to_s == reflection.foreign_key.to_s && belongs_to.options[:counter_cache]+ end.counter_cache_column++ counts = reflection.klass.where(reflection.foreign_key => ids).group(reflection.foreign_key).count++ ids.group_by { |id| counts.fetch(id, 0) }.each do |count, grouped_ids|+ unscoped.where(id: grouped_ids).where.not(counter => count).update_all(counter => count)+ end+ end+# @see Spree.mysql? — the single definition; this reads it so a model can# branch without reaching for the connection itself.def mysql_adapter?diff --git a/spree/core/app/models/spree/cart.rb b/spree/core/app/models/spree/cart.rbindex 2ca557ec31..3e705e70e2 100644--- a/spree/core/app/models/spree/cart.rb+++ b/spree/core/app/models/spree/cart.rb@@ -203,10 +203,13 @@ module Spree# delivery_unavailable warning per affected line item (parity with the# order-side ensure_available_delivery_rates).def prune_undeliverable_fulfillments!- undeliverable = fulfillments.reload.select { |fulfillment| fulfillment.delivery_rates.empty? }+ candidates = fulfillments.reload.to_a+ ActiveRecord::Associations::Preloader.new(records: candidates, associations: :delivery_rates).call+ undeliverable = candidates.select { |fulfillment| fulfillment.delivery_rates.empty? }return if undeliverable.empty?affected_line_items = undeliverable.flat_map(&:line_items).uniq+ ActiveRecord::Associations::Preloader.new(records: affected_line_items, associations: { variant: :product }).callundeliverable.each(&:destroy)self.warnings |= affected_line_items.map do |line_item|{diff --git a/spree/core/app/models/spree/category.rb b/spree/core/app/models/spree/category.rbindex 986589c490..2aa5ff08a6 100644--- a/spree/core/app/models/spree/category.rb+++ b/spree/core/app/models/spree/category.rb@@ -81,7 +81,13 @@ module Spreevalidates :meta_descriptionvalidates :meta_titleend- validates :image, :square_image, content_type: Rails.application.config.active_storage.web_image_content_types+ # Only a file assigned in this write is checked: a stored one passed when+ # it was attached, and reading it back would cost a query per slot on+ # every save — including each category a product write validates.+ validates :image, content_type: Rails.application.config.active_storage.web_image_content_types,+ if: -> { attachment_changes.key?('image') }+ validates :square_image, content_type: Rails.application.config.active_storage.web_image_content_types,+ if: -> { attachment_changes.key?('square_image') }## Callbacks@@ -183,16 +189,81 @@ module Spreedef self.recalculate_products_count(category_ids)changed = unscoped.where(id: Array(category_ids).compact.uniq)- # The affected nodes are each changed category plus its ancestors.- affected = changed.flat_map { |category| category.self_and_ancestors.to_a }.uniq(&:id)+ # The affected nodes are each changed category plus its ancestors,+ # collected in one read: an ancestor chain per category is a query per+ # node, which a bulk catalog move pays for every category it touched.+ # A nested set makes an ancestor a node whose `lft`/`rgt` span the+ # child's, so one range query answers for all of them.+ changed = changed.to_a+ return if changed.empty?++ ancestor_conditions = changed.map do |category|+ arel_table[:store_id].eq(category.store_id).+ and(arel_table[:lft].lteq(category.lft)).+ and(arel_table[:rgt].gteq(category.rgt))+ end.reduce(:or)++ affected = unscoped.where(ancestor_conditions).to_a++ # One grouped query for the whole set rather than a count per node:+ # this runs on every catalog move, and a deep tree would otherwise+ # spend a query on each affected ancestor.+ counts = distinct_product_counts_for(affected)affected.each do |category|- count = Spree::ProductCategory.where(category_id: category.self_and_descendants.select(:id))- .distinct.count(:product_id)+ count = counts.fetch(category.id, 0)category.update_column(:products_count, count) if category.products_count != countendend+ # Distinct product counts per category, each counting its whole subtree.+ # A nested set stores the subtree as an `lft`/`rgt` range, so the tie+ # between a node and its descendants is a range join rather than a+ # separate descendant query for every node.+ #+ # @param categories [Array<Spree::Category>]+ # @return [Hash{Integer => Integer}] category id => distinct product count+ def self.distinct_product_counts_for(categories)+ return {} if categories.empty?++ nodes = arel_table.alias('counted_nodes')+ descendants = arel_table+ product_categories = Spree::ProductCategory.arel_table++ relation = unscoped.+ from(nodes).+ joins(+ Arel::Nodes::InnerJoin.new(+ descendants,+ Arel::Nodes::On.new(+ descendants[:store_id].eq(nodes[:store_id]).+ and(descendants[:lft].gteq(nodes[:lft])).+ and(descendants[:rgt].lteq(nodes[:rgt]))+ )+ )+ ).+ joins(+ Arel::Nodes::InnerJoin.new(+ product_categories,+ Arel::Nodes::On.new(product_categories[:category_id].eq(descendants[:id]))+ )+ ).+ where(nodes[:id].in(categories.map(&:id))).+ group(nodes[:id])++ relation.distinct.count(product_categories[:product_id])+ end++ # The breadcrumb trail, with the images a serializer renders alongside+ # each crumb already loaded. `ancestors` comes from the nested set and+ # preloads nothing, so drawing a trail would otherwise cost two+ # attachment lookups per level.+ #+ # @return [ActiveRecord::Relation<Spree::Category>]+ def ancestors+ super.with_attached_image.with_attached_square_image+ end+def slugpermalinkenddiff --git a/spree/core/app/models/spree/channel.rb b/spree/core/app/models/spree/channel.rbindex 1fde2a6072..c66b967abd 100644--- a/spree/core/app/models/spree/channel.rb+++ b/spree/core/app/models/spree/channel.rb@@ -62,6 +62,47 @@ module Spreeself.whitelisted_ransackable_attributes = %w[name code active default store_id]search_by :name+ # Publishes the given products on every channel in the current scope+ # (`store.channels.where(id: ids).add_products(product_ids)`), touching+ # and reindexing each product once rather than once per channel.+ # @param product_ids [Array<Integer>, Integer]+ # @return [Integer] the number of ProductPublications created or updated+ def self.add_products(product_ids)+ product_ids = Array(product_ids).map(&:to_s).uniq+ channels = all.to_a+ return 0 if product_ids.empty? || channels.empty?++ count = channels.sum { |channel| channel.send(:upsert_publications, product_ids) }+ refresh_products(product_ids)+ channels.each(&:touch)+ count+ end++ # Unpublishes the given products from every channel in the current scope.+ # @param product_ids [Array<Integer>, Integer]+ # @return [Integer] the number of ProductPublications destroyed+ def self.remove_products(product_ids)+ product_ids = Array(product_ids).map(&:to_s).uniq+ channels = all.to_a+ return 0 if product_ids.empty? || channels.empty?++ removed = Spree::ProductPublication.includes(:product).where(channel_id: channels.map(&:id), product_id: product_ids).destroy_all+ refresh_products(product_ids)+ emptied_channel_ids = removed.map(&:channel_id).uniq+ channels.select { |channel| emptied_channel_ids.include?(channel.id) }.each(&:touch)+ removed.size+ end++ # @param product_ids [Array<String>]+ # @return [void]+ def self.refresh_products(product_ids)+ products = Spree::Product.where(id: product_ids)+ products.touch_all+ # `to_a` once: `touch_all` does not load the relation, so iterating it+ # afterwards would re-run the query.+ products.to_a.each(&:enqueue_search_index)+ end+# Publishes the given products on this channel by creating/upserting ProductPublications.# Optionally sets the publication window; if not given, the products will be published immediately# with no end date.@@ -73,42 +114,10 @@ module Spreeproduct_ids = Array(product_ids).map(&:to_s).uniqreturn 0 if product_ids.empty?- now = Time.current- # Only include window columns in the upsert payload when the caller- # explicitly passed a value. Leaving them out keeps existing- # publication schedules intact on re-publish — otherwise +on_duplicate:- # :update+ + +update_only+ would rewrite scheduled +published_at+ /- # +unpublished_at+ to NULL whenever the bulk action re-runs without- # dates.- base = { channel_id: id, created_at: now, updated_at: now }- base[:published_at] = published_at unless published_at.nil?- base[:unpublished_at] = unpublished_at unless unpublished_at.nil?-- records_to_upsert = product_ids.map { |product_id| base.merge(product_id: product_id) }-- # Only update the window columns the caller passed. When neither was- # passed, treat re-publish as a no-op (+on_duplicate: :skip+ → MySQL- # +INSERT IGNORE+, PG/SQLite +ON CONFLICT DO NOTHING+).- update_columns = []- update_columns << :published_at unless published_at.nil?- update_columns << :unpublished_at unless unpublished_at.nil?- opts = if update_columns.empty?- { on_duplicate: :skip }- else- { record_timestamps: false, update_only: update_columns, on_duplicate: :update }- end- # MySQL infers the conflict target from the table's unique constraints- # and rejects an explicit +unique_by+; PostgreSQL/SQLite require it.- opts[:unique_by] = %i[product_id channel_id] unless mysql_adapter?-- Spree::ProductPublication.upsert_all(records_to_upsert, **opts)-- products = Spree::Product.where(id: product_ids)- products.touch_all- products.each(&:enqueue_search_index)+ count = upsert_publications(product_ids, published_at: published_at, unpublished_at: unpublished_at)+ self.class.refresh_products(product_ids)touch-- records_to_upsert.size+ countend# Unpublishes the given products from this channel.@@ -118,12 +127,9 @@ module Spreeproduct_ids = Array(product_ids).map(&:to_s).uniqreturn 0 if product_ids.empty?- count = publications.where(product_id: product_ids).destroy_all.size-- products = Spree::Product.where(id: product_ids)- products.touch_all- products.each(&:enqueue_search_index)+ count = publications.includes(:product).where(product_id: product_ids).destroy_all.size+ self.class.refresh_products(product_ids)touch if count.positive?countend@@ -161,6 +167,39 @@ module Spreeprivate+ def upsert_publications(product_ids, published_at: nil, unpublished_at: nil)+ now = Time.current+ # Only include window columns in the upsert payload when the caller+ # explicitly passed a value. Leaving them out keeps existing+ # publication schedules intact on re-publish — otherwise +on_duplicate:+ # :update+ + +update_only+ would rewrite scheduled +published_at+ /+ # +unpublished_at+ to NULL whenever the bulk action re-runs without+ # dates.+ base = { channel_id: id, created_at: now, updated_at: now }+ base[:published_at] = published_at unless published_at.nil?+ base[:unpublished_at] = unpublished_at unless unpublished_at.nil?++ records_to_upsert = product_ids.map { |product_id| base.merge(product_id: product_id) }++ # Only update the window columns the caller passed. When neither was+ # passed, treat re-publish as a no-op (+on_duplicate: :skip+ → MySQL+ # +INSERT IGNORE+, PG/SQLite +ON CONFLICT DO NOTHING+).+ update_columns = []+ update_columns << :published_at unless published_at.nil?+ update_columns << :unpublished_at unless unpublished_at.nil?+ opts = if update_columns.empty?+ { on_duplicate: :skip }+ else+ { record_timestamps: false, update_only: update_columns, on_duplicate: :update }+ end+ # MySQL infers the conflict target from the table's unique constraints+ # and rejects an explicit +unique_by+; PostgreSQL/SQLite require it.+ opts[:unique_by] = %i[product_id channel_id] unless mysql_adapter?++ Spree::ProductPublication.upsert_all(records_to_upsert, **opts)+ records_to_upsert.size+ end+def ensure_not_defaultreturn unless default?return if destroyed_by_association.present?diff --git a/spree/core/app/models/spree/company.rb b/spree/core/app/models/spree/company.rbindex b841ae3e9a..3f1ace8b07 100644--- a/spree/core/app/models/spree/company.rb+++ b/spree/core/app/models/spree/company.rb@@ -177,10 +177,47 @@ module Spreeself_and_descendants.where.not(id: id)end+ # Distinct members across each node's whole subtree, for many nodes in+ # one query.+ #+ # @param company_ids [Array] the subtree roots+ # @return [Hash{Integer => Integer}] member count by root id; roots with none are absent+ def self.members_counts_for(company_ids)+ return {} if company_ids.empty?++ membership_table = Spree::CompanyMembership.table_name+ subtree_sql = sanitize_sql_array([<<~SQL.squish, company_ids])+ WITH RECURSIVE subtree(root_id, id) AS (+ SELECT id, id FROM #{table_name} WHERE id IN (?)+ UNION+ SELECT subtree.root_id, child.id FROM #{table_name} child+ INNER JOIN subtree ON child.parent_id = subtree.id+ )+ SELECT root_id, id FROM subtree+ SQL++ Spree::CompanyMembership.+ joins("INNER JOIN (#{subtree_sql}) company_subtree ON company_subtree.id = #{membership_table}.company_id").+ group('company_subtree.root_id').+ distinct.count(:customer_id)+ end++ # On a lazily preloaded list, the first call counts every node of the+ # list at once.+ ## @return [Integer]def members_count- Spree::CompanyMembership.where(company_id: self_and_descendants.select(:id)).- distinct.count(:customer_id)+ return @members_count if defined?(@members_count)++ siblings = lazy_preload_context&.records&.grep(self.class)+ if siblings.blank?+ return Spree::CompanyMembership.where(company_id: self_and_descendants.select(:id)).+ distinct.count(:customer_id)+ end++ counts = self.class.members_counts_for(siblings.map(&:id))+ siblings.each { |company| company.preloaded_members_count = counts.fetch(company.id, 0) }+ @members_countend# The single tax anchor: the nearest self-or-ancestor +company+ node. Tax@@ -248,6 +285,12 @@ module SpreeSpree::Address.not_deleted.where(owner_type: 'Spree::Company', owner_id: self_and_ancestors.map(&:id))end+ protected++ def preloaded_members_count=(count)+ @members_count = count+ end+privatedef ensure_can_be_deleteddiff --git a/spree/core/app/models/spree/customer_group.rb b/spree/core/app/models/spree/customer_group.rbindex d9f4b2057b..8ad0fc750e 100644--- a/spree/core/app/models/spree/customer_group.rb+++ b/spree/core/app/models/spree/customer_group.rb@@ -21,7 +21,10 @@ module Spree# own association (rather than `alias_method`) is what lets `customer_ids=`# exist and what makes the `PrefixedId` auto-decoder in `assign_attributes`# recognise the key — that lookup keys off `reflect_on_association(:customers)`.- has_many :customers, through: :customer_group_users, source: :customer, source_type: Spree.customer_class.to_s+ # Joining a group changes nothing on the customer, so saving the group+ # does not re-validate every member it lists.+ has_many :customers, through: :customer_group_users, source: :customer, source_type: Spree.customer_class.to_s,+ validate: false## Validations@@ -41,64 +44,79 @@ module Spree# @param user_ids [Array] array of user IDs to add# @return [Integer] number of customers addeddef add_customers(user_ids)- return 0 if user_ids.blank?+ self.class.add_customers_to([self], user_ids)+ end- user_ids = Array(user_ids).map(&:to_s).uniq- return 0 if user_ids.empty?+ # Bulk remove customers from the group+ # @param user_ids [Array] array of user IDs to remove+ # @return [Integer] number of customers removed+ def remove_customers(user_ids)+ self.class.remove_customers_from([self], user_ids)+ end- # Get existing user IDs to avoid duplicates- existing_user_ids = customer_group_users.where(customer_id: user_ids).pluck(:customer_id).map(&:to_s).to_set+ # Adds customers to every given group, skipping memberships that already+ # exist, with one lookup for all groups.+ # @param groups [Array<Spree::CustomerGroup>]+ # @param customer_ids [Array] customer IDs to add+ # @return [Integer] number of memberships created+ def self.add_customers_to(groups, customer_ids)+ customer_ids = Array(customer_ids).compact_blank.map(&:to_s).uniq+ groups = Array(groups)+ return 0 if customer_ids.empty? || groups.empty?++ existing_pairs = Spree::CustomerGroupUser.+ where(customer_group_id: groups.map(&:id), customer_id: customer_ids).+ pluck(:customer_group_id, :customer_id).+ to_set { |group_id, customer_id| [group_id, customer_id.to_s] }now = Time.current- user_type = Spree.customer_class.to_s-- records_to_insert = user_ids.filter_map do |user_id|- next if existing_user_ids.include?(user_id)-- {- customer_group_id: id,- user_id: user_id,- user_type: user_type,- created_at: now,- updated_at: now- }+ customer_type = Spree.customer_class.to_s+ records_to_insert = groups.flat_map do |group|+ customer_ids.filter_map do |customer_id|+ next if existing_pairs.include?([group.id, customer_id])++ {+ customer_group_id: group.id,+ customer_id: customer_id,+ customer_type: customer_type,+ created_at: now,+ updated_at: now+ }+ endend-return 0 if records_to_insert.empty?Spree::CustomerGroupUser.upsert_all(records_to_insert, on_duplicate: :skip)- added_user_ids = records_to_insert.map { |r| r[:user_id] }- touch_users(added_user_ids)- touch+ touch_customers(records_to_insert.map { |record| record[:customer_id] }.uniq)+ changed_group_ids = records_to_insert.map { |record| record[:customer_group_id] }.to_set+ groups.select { |group| changed_group_ids.include?(group.id) }.each(&:touch)records_to_insert.sizeend- # Bulk remove customers from the group- # @param user_ids [Array] array of user IDs to remove- # @return [Integer] number of customers removed- def remove_customers(user_ids)- return 0 if user_ids.blank?+ # Removes customers from every given group.+ # @param groups [Array<Spree::CustomerGroup>]+ # @param customer_ids [Array] customer IDs to remove+ # @return [Integer] number of memberships removed+ def self.remove_customers_from(groups, customer_ids)+ customer_ids = Array(customer_ids).compact_blank.map(&:to_s).uniq+ groups = Array(groups)+ return 0 if customer_ids.empty? || groups.empty?- user_ids = Array(user_ids).map(&:to_s).uniq- return 0 if user_ids.empty?+ memberships = Spree::CustomerGroupUser.where(customer_group_id: groups.map(&:id), customer_id: customer_ids)+ changed_group_ids = memberships.distinct.pluck(:customer_group_id).to_set+ return 0 if changed_group_ids.empty?- deleted_count = customer_group_users.where(customer_id: user_ids).delete_all-- if deleted_count > 0- touch_users(user_ids)- touch- end+ deleted_count = memberships.delete_all+ touch_customers(customer_ids)+ groups.select { |group| changed_group_ids.include?(group.id) }.each(&:touch)deleted_countend- private-- def touch_users(user_ids)- return if user_ids.blank?-- Spree.customer_class.where(id: user_ids).touch_all+ def self.touch_customers(customer_ids)+ Spree.customer_class.where(id: customer_ids).touch_allend+ private_class_method :touch_customersendenddiff --git a/spree/core/app/models/spree/delivery_method.rb b/spree/core/app/models/spree/delivery_method.rbindex db3ec82b9e..b625c568ee 100644--- a/spree/core/app/models/spree/delivery_method.rb+++ b/spree/core/app/models/spree/delivery_method.rb@@ -500,14 +500,21 @@ module Spreereconcile_services(pending)end+ # Matched against the method's rows loaded once, rather than a lookup per+ # row. A row created here joins that list, so a later row naming the+ # same (carrier, service) updates it as a lookup would.def reconcile_services(rows)+ existing = services.reload.to_akept_ids = rows.filter_map do |row|- record =- if row[:id].present?- id = Spree::PrefixedId.prefixed_id?(row[:id]) ? Spree::PrefixedId.decode_prefixed_id(row[:id]) : row[:id]- services.find_by(id: id)- end- record ||= services.find_or_initialize_by(carrier: row[:carrier], service: row[:service])+ if row[:id].present?+ id = Spree::PrefixedId.prefixed_id?(row[:id]) ? Spree::PrefixedId.decode_prefixed_id(row[:id]) : row[:id]+ record = existing.find { |service| service.id.to_s == id.to_s }+ end+ record ||= existing.find { |service| service.carrier == row[:carrier] && service.service == row[:service] }+ unless record+ record = services.build(carrier: row[:carrier], service: row[:service])+ existing << record+ endrecord.assign_attributes(row.slice(:carrier, :service, :label, :markup_flat, :markup_percent, :position).to_h)record.save!record.iddiff --git a/spree/core/app/models/spree/delivery_profile.rb b/spree/core/app/models/spree/delivery_profile.rbindex d782c13d5f..a4ee6ff9f8 100644--- a/spree/core/app/models/spree/delivery_profile.rb+++ b/spree/core/app/models/spree/delivery_profile.rb@@ -123,6 +123,24 @@ module Spreedelivery_origin_groups.firstend+ # How many products use this profile. Counted for every profile of a+ # lazily preloaded list in one grouped query, since loading the products+ # themselves to count them would fetch the whole catalog.+ #+ # @return [Integer]+ def products_count+ return @products_count if defined?(@products_count)++ profiles = lazy_preload_context&.records.presence || [self]+ counts = Spree::Product.where(delivery_profile_id: profiles.map(&:id)).group(:delivery_profile_id).count+ profiles.each { |profile| profile.products_count = counts.fetch(profile.id, 0) }+ @products_count+ end++ protected++ attr_writer :products_count+privatedef create_default_origin_groupdiff --git a/spree/core/app/models/spree/fulfillment.rb b/spree/core/app/models/spree/fulfillment.rbindex 78a7917f9e..f2a9deaf44 100644--- a/spree/core/app/models/spree/fulfillment.rb+++ b/spree/core/app/models/spree/fulfillment.rb@@ -653,7 +653,14 @@ module Spree## @return [Spree::DeliveryMethod]def delivery_method- selected_delivery_rate&.delivery_method || delivery_rates.first&.delivery_method+ selected = selected_delivery_rate+ # The same row read through the rate list, when that list is to hand,+ # so its method loads with every other rate's instead of on its own.+ if selected && read_association_in_memory?(:delivery_rates)+ selected = delivery_rates.detect { |rate| rate.id == selected.id } || selected+ end++ selected&.delivery_method || delivery_rates.first&.delivery_methodend# The fulfillment provider strategy handling this fulfillment'sdiff --git a/spree/core/app/models/spree/fulfillment_changer.rb b/spree/core/app/models/spree/fulfillment_changer.rbindex fd402341e1..606f0b7712 100644--- a/spree/core/app/models/spree/fulfillment_changer.rb+++ b/spree/core/app/models/spree/fulfillment_changer.rb@@ -138,7 +138,9 @@ module Spreeenddef reload_shipment_inventory_units- [current_shipment, desired_shipment].each { |shipment| shipment.fulfillment_items.reload }+ shipments = [current_shipment, desired_shipment]+ shipments.each { |shipment| shipment.association(:fulfillment_items).reset }+ ActiveRecord::Associations::Preloader.new(records: shipments, associations: :fulfillment_items).callenddef current_quantitydiff --git a/spree/core/app/models/spree/fulfillment_item.rb b/spree/core/app/models/spree/fulfillment_item.rbindex 3d2c0e3c46..7a5625f002 100644--- a/spree/core/app/models/spree/fulfillment_item.rb+++ b/spree/core/app/models/spree/fulfillment_item.rb@@ -107,6 +107,17 @@ module Spreesplitend+ # `dup` copies the foreign keys but not the loaded records, so every copy+ # made while packing and splitting would look its line item and variant+ # up again.+ def initialize_dup(other)+ super+ %i[line_item variant].each do |name|+ association = other.association(name)+ self.association(name).target = association.target if association.loaded?+ end+ end+# This will fail if extract >= available_quantitydef split_inventory!(extract_quantity)split = self.class.split(self, extract_quantity)diff --git a/spree/core/app/models/spree/import.rb b/spree/core/app/models/spree/import.rbindex 37a34da086..dcac1ce801 100644--- a/spree/core/app/models/spree/import.rb+++ b/spree/core/app/models/spree/import.rb@@ -215,10 +215,22 @@ module Spreeend# Per-status row counts for API status payloads, memoized so one request- # issues a single grouped COUNT instead of one per status.+ # issues a single grouped COUNT instead of one per status — and, on a+ # lazily preloaded list, one for every import of the list.# @return [Hash{String => Integer}]def rows_status_counts- @rows_status_counts ||= rows.group(:status).count+ return @rows_status_counts if @rows_status_counts++ siblings = lazy_preload_context&.records&.grep(Spree::Import)+ return @rows_status_counts = rows.group(:status).count if siblings.blank?++ counts = Spree::ImportRow.where(import_id: siblings.map(&:id)).group(:import_id, :status).count+ siblings.each do |import|+ import.preloaded_rows_status_counts = counts.each_with_object({}) do |((import_id, status), count), by_status|+ by_status[status] = count if import_id == import.id+ end+ end+ @rows_status_countsend# Public API name for the +results_url+ preference (read/write symmetry).@@ -474,6 +486,12 @@ module Spreeendend+ protected++ def preloaded_rows_status_counts=(counts)+ @rows_status_counts = counts+ end+private# Mirrors the machine's bang events: an illegal move raised.diff --git a/spree/core/app/models/spree/order_routing/strategy/rules.rb b/spree/core/app/models/spree/order_routing/strategy/rules.rbindex b17d7139c6..739b09ec53 100644--- a/spree/core/app/models/spree/order_routing/strategy/rules.rb+++ b/spree/core/app/models/spree/order_routing/strategy/rules.rb@@ -59,8 +59,11 @@ module Spree# rank order so the Prioritizer's first-package-wins-on-hand logic# honors the routing decision.def build_packages(locations)+ stock_levels = Spree::Stock::Packer.stock_levels_by_location(locations, requested_variant_ids)+locations.flat_map do |location|- Spree::Stock::Packer.new(location, inventory_units, Spree.stock_splitters, owner: order).packages+ Spree::Stock::Packer.new(location, inventory_units, Spree.stock_splitters, owner: order,+ stock_levels: stock_levels.fetch(location.id, {})).packagesendenddiff --git a/spree/core/app/models/spree/price_list.rb b/spree/core/app/models/spree/price_list.rbindex 9a52dadb47..65f0311119 100644--- a/spree/core/app/models/spree/price_list.rb+++ b/spree/core/app/models/spree/price_list.rb@@ -38,6 +38,21 @@ module Spree# flip or a catalog attach/detach changes which lists match generically.after_commit -> { Spree::Current.price_lists = nil }+ # Distinct products per list for a page of lists, in one grouped query.+ # @param price_list_ids [Array<Integer>]+ # @return [Hash{Integer => Integer}] keyed by price list id; lists with no products are absent+ def self.products_count_by_list(price_list_ids)+ where(id: price_list_ids).joins(:variants).group(arel_table[:id]).+ distinct.count(Spree::Variant.arel_table[:product_id])+ end++ # Prices per list for a page of lists, in one grouped query.+ # @param price_list_ids [Array<Integer>]+ # @return [Hash{Integer => Integer}] keyed by price list id; lists with no prices are absent+ def self.prices_count_by_list(price_list_ids)+ Spree::Price.where(price_list_id: price_list_ids).group(:price_list_id).count+ end+# Flat-payload writer for `rules`. See# {Spree::TypedAssociations#assign_typed_association}.def rules=(rows)diff --git a/spree/core/app/models/spree/product.rb b/spree/core/app/models/spree/product.rbindex f0528d3c2a..bbacb99d6b 100644--- a/spree/core/app/models/spree/product.rb+++ b/spree/core/app/models/spree/product.rb@@ -497,7 +497,9 @@ module Spreedef purchasable?return @purchasable unless @purchasable.nil?- @purchasable = sellable_variants.any?(&:purchasable?)+ variants_to_check = sellable_variants+ Spree::Stock::Quantifier.preload(variants_to_check)+ @purchasable = variants_to_check.any?(&:purchasable?)enddef in_stock?diff --git a/spree/core/app/models/spree/product/channels.rb b/spree/core/app/models/spree/product/channels.rbindex 3595bbc710..655a321386 100644--- a/spree/core/app/models/spree/product/channels.rb+++ b/spree/core/app/models/spree/product/channels.rb@@ -64,8 +64,8 @@ module Spreedef publication_for(channel)return nil unless channel- if product_publications.loaded?- product_publications.find { |p| p.channel_id == channel.id }+ if read_association_in_memory?(:product_publications)+ product_publications.find { |publication| publication.channel_id == channel.id }elseproduct_publications.find_by(channel_id: channel.id)end@@ -122,10 +122,16 @@ module Spreeenddef apply_product_publications(publications_params)+ rows = publications_params.map { |publication_data| publication_data.to_h.with_indifferent_access }publication_ids_in_payload = []- publications_params.each do |publication_data|- publication_data = publication_data.to_h.with_indifferent_access+ # The product's publications and every channel the payload names are+ # read once, rather than a lookup per row.+ named_channel_ids = rows.filter_map { |row| decode_publication_channel_id(row[:channel_id]) if row[:id].blank? }+ channels_by_id = named_channel_ids.empty? ? {} : publishable_channels.where(id: named_channel_ids).index_by { |channel| channel.id.to_s }+ product_publications.load++ rows.each do |publication_data|publication_id = publication_data.delete(:id)channel_id = decode_publication_channel_id(publication_data[:channel_id])@@ -133,7 +139,7 @@ module Spreedecoded_id = Spree::PrefixedId.prefixed_id?(publication_id) ?Spree::PrefixedId.decode_prefixed_id(publication_id) :publication_id- publication = product_publications.find_by(id: decoded_id)+ publication = product_publications.detect { |existing| existing.id.to_s == decoded_id.to_s }next unless publication# Channel is immutable; ignore any rebind attempt.@@ -142,19 +148,21 @@ module Spreeelsif channel_id.present?# Only publish to channels owned by the product's store; ignore# ids referencing another store's channel.- channel = publishable_channels.find_by(id: channel_id)+ channel = channels_by_id[channel_id.to_s]next unless channel# Upsert by channel_id so repeat submissions are idempotent# against the unique (product_id, channel_id) index.- publication = product_publications.find_or_initialize_by(channel_id: channel.id)+ publication = product_publications.detect { |existing| existing.channel_id == channel.id } ||+ product_publications.build(channel: channel)publication.assign_attributes(publication_data.slice(:published_at, :unpublished_at))publication.save!publication_ids_in_payload << publication.idendend- product_publications.where.not(id: publication_ids_in_payload).destroy_all+ removed = product_publications.reject { |publication| publication_ids_in_payload.include?(publication.id) }+ product_publications.destroy(*removed) if removed.any?enddef decode_publication_channel_id(value)diff --git a/spree/core/app/models/spree/return.rb b/spree/core/app/models/spree/return.rbindex ea880bdfe8..4c797a5a2e 100644--- a/spree/core/app/models/spree/return.rb+++ b/spree/core/app/models/spree/return.rb@@ -165,7 +165,7 @@ module Spree# more than one step (part to store credit, part to the card), and store# credit is its own ledger rather than a Spree::Refund row.def refunded_total- refunds.sum(:amount) + store_credits.sum(:amount)+ ledger_total(:refunds) + ledger_total(:store_credits)enddef refundable_total@@ -187,5 +187,12 @@ module Spreedef taxed_linesreturn_line_itemsend++ private++ def ledger_total(association_name)+ rows = public_send(association_name)+ read_association_in_memory?(association_name) ? rows.sum(0.to_d, &:amount) : rows.sum(:amount)+ endendenddiff --git a/spree/core/app/models/spree/seller.rb b/spree/core/app/models/spree/seller.rbindex 073587cc80..86430c7d14 100644--- a/spree/core/app/models/spree/seller.rb+++ b/spree/core/app/models/spree/seller.rb@@ -310,10 +310,7 @@ module Spree## @return [Array<Spree::SellerBalance>]def balances- pairs = seller_transfers.where.not(status: 'failed').- distinct.pluck(:currency, :settled_currency)-- pairs.sort.map { |currency, settlement| Spree::SellerBalance.for(self, currency, settlement) }+ Spree::SellerBalance.all_for(self)end# The seller's account with whichever provider pays them — a Stripe Connect@@ -399,6 +396,10 @@ module Spree# returns; falls back to any active one, since a seller with none marked# still has to receive what customers send back.def returns_location+ if stock_locations.loaded? || lazy_preload_context+ return @returns_location ||= returns_location_from_loaded+ end+@returns_location ||= stock_locations.active.returns_enabled.order_default.first ||stock_locations.active.order_default.firstend@@ -417,9 +418,24 @@ module Spreelocation.addressend+ # A team is small, so on a list its members are loaded for every seller+ # at once rather than counted one seller at a time.+ #+ # @return [Integer] how many people are on this seller's team+ def users_count+ users.loaded? || lazy_preload_context ? users.to_a.size : users.count+ end+# @return [Integer] how many products this seller listsdef products_count- @products_count ||= products.count+ return @products_count if @products_count++ siblings = lazy_preload_context&.records&.grep(self.class)+ return @products_count = products.count if siblings.blank?++ counts = Spree::Product.where(seller_id: siblings.map(&:id)).group(:seller_id).count+ siblings.each { |seller| seller.preloaded_products_count = counts.fetch(seller.id, 0) }+ @products_countend# Where this seller stands against the marketplace's checklist@@ -465,8 +481,22 @@ module Spreeonboarding_requirements.none?(&:blocking?)end+ protected++ def preloaded_products_count=(count)+ @products_count = count+ end+private+ # The same choice as the query in {#returns_location}, made over the+ # preloaded locations.+ def returns_location_from_loaded+ active_locations = stock_locations.select(&:active?).sort_by { |location| [location.default? ? 0 : 1, location.name.to_s] }++ active_locations.find(&:returns_enabled?) || active_locations.first+ end+def normalize_slugself.slug = (slug.presence || name).to_s.parameterize.presencediff --git a/spree/core/app/models/spree/seller_balance.rb b/spree/core/app/models/spree/seller_balance.rbindex be07532113..4cdc2a1a2d 100644--- a/spree/core/app/models/spree/seller_balance.rb+++ b/spree/core/app/models/spree/seller_balance.rb@@ -57,6 +57,34 @@ module Spree)end+ # Every position the seller holds, read with one grouped query per figure+ # rather than four per position. See {Spree::Seller#balances}.+ #+ # @param seller [Spree::Seller]+ # @return [Array<Spree::SellerBalance>]+ def self.all_for(seller)+ transfers = seller.seller_transfers.where.not(status: 'failed')+ amounts = transfers.group(:currency, :settled_currency, :status).sum(:amount)+ payable = transfers.completed.group(:currency, :settled_currency).sum(:settled_amount)+ paid = transfers.completed.joins(:payout).merge(Spree::SellerPayout.completed).+ group(:currency, :settled_currency).sum(:settled_amount)++ pairs = amounts.keys.map { |currency, settlement_currency, _status| [currency, settlement_currency] }.uniq.sort+ pairs.map do |pair|+ amount_in = ->(*statuses) { statuses.sum(BigDecimal('0')) { |status| amounts.fetch([*pair, status], 0) } }++ new(+ seller: seller,+ currency: pair.first,+ settlement_currency: pair.last,+ earned: amount_in.call('completed'),+ pending: amount_in.call('pending', 'processing', 'unresolved'),+ payable: payable.fetch(pair, 0),+ paid: paid.fetch(pair, 0)+ )+ end+ end+# What the marketplace still owes, in the currency it can actually be sent# in.#diff --git a/spree/core/app/models/spree/seller_requirements/delivery_method.rb b/spree/core/app/models/spree/seller_requirements/delivery_method.rbindex 0d7cdf440b..1e4be5d103 100644--- a/spree/core/app/models/spree/seller_requirements/delivery_method.rb+++ b/spree/core/app/models/spree/seller_requirements/delivery_method.rb@@ -11,9 +11,27 @@ module Spree# has shared at least one of theirs.class DeliveryMethod < Spree::SellerRequirementdef met_by_seller?(seller)- return true if seller.delivery_methods.storefront_visible.exists?+ return true if ships_own_methods?(seller)- store.delivery_methods.shared_with_sellers.storefront_visible.exists?+ operator_shares_methods?+ end++ private++ def ships_own_methods?(seller)+ if seller.read_association_in_memory?(:delivery_methods)+ seller.delivery_methods.any?(&:storefront_visible?)+ else+ seller.delivery_methods.storefront_visible.exists?+ end+ end++ # The same answer for every seller of the store, so a list of them+ # asks once.+ def operator_shares_methods?+ return @operator_shares_methods if defined?(@operator_shares_methods)++ @operator_shares_methods = store.delivery_methods.shared_with_sellers.storefront_visible.exists?endendenddiff --git a/spree/core/app/models/spree/stock/coordinator.rb b/spree/core/app/models/spree/stock/coordinator.rbindex 7b94f7b4df..1b7e6aea18 100644--- a/spree/core/app/models/spree/stock/coordinator.rb+++ b/spree/core/app/models/spree/stock/coordinator.rb@@ -28,11 +28,14 @@ module Spreeenddef build_packages(packages = [])- stock_locations_with_requested_variants.each do |stock_location|+ stock_locations = stock_locations_with_requested_variants.to_a+ stock_levels = Packer.stock_levels_by_location(stock_locations, requested_variant_ids)++ stock_locations.each do |stock_location|units = allocatable_units_for(stock_location)next if units.empty?- packer = build_packer(stock_location, units)+ packer = build_packer(stock_location, units, stock_levels: stock_levels.fetch(stock_location.id, {}))packages += packer.packagesend@@ -114,8 +117,8 @@ module Spree@estimator ||= Estimator.new(order)end- def build_packer(stock_location, inventory_units)- Packer.new(stock_location, inventory_units, splitters(stock_location), owner: order)+ def build_packer(stock_location, inventory_units, stock_levels: nil)+ Packer.new(stock_location, inventory_units, splitters(stock_location), owner: order, stock_levels: stock_levels)enddef splitters(_stock_location)diff --git a/spree/core/app/models/spree/stock/estimator.rb b/spree/core/app/models/spree/stock/estimator.rbindex 89d26101e4..99612e1d37 100644--- a/spree/core/app/models/spree/stock/estimator.rb+++ b/spree/core/app/models/spree/stock/estimator.rb@@ -162,6 +162,12 @@ module Spreemethods = package.eligible_delivery_methods# Storeless orders exist only in specs; real orders always carry one.methods = methods.merge(order.store.delivery_methods) if order.store+ # Read for every method below and when its rates are built. Through the+ # preloader because an overridden package may hand back an Array.+ methods = methods.to_a+ ActiveRecord::Associations::Preloader.new(+ records: methods, associations: %i[delivery_method_rules calculator services delivery_zone]+ ).call# On a marketplace, a package's seller is its stock location's, and it# is quoted by that seller's own methods plus whatever the operatordiff --git a/spree/core/app/models/spree/stock/inventory_unit_builder.rb b/spree/core/app/models/spree/stock/inventory_unit_builder.rbindex 6675fafb18..93027bf5dc 100644--- a/spree/core/app/models/spree/stock/inventory_unit_builder.rb+++ b/spree/core/app/models/spree/stock/inventory_unit_builder.rb@@ -11,16 +11,28 @@ module Spree# rather than pointing at whatever order happens to share the cart's# id — a stranger's order whose currency and ship address would then# stand in for the customer's.- @order.line_items.map do |line_item|+ line_items = @order.line_items.to_a+ # Allocation asks each variant how it ships, so the variants and their+ # delivery profiles are read once for the whole cart.+ ActiveRecord::Associations::Preloader.new(+ records: line_items, associations: { variant: [:delivery_profile, { product: :delivery_profile }] }+ ).call++ line_items.map do |line_item|# They go through multiple splits, avoid loading the# association to order until needed.- Spree::FulfillmentItem.new(+ unit = Spree::FulfillmentItem.new(pending: true,line_item_id: line_item.id,variant_id: line_item.variant_id,quantity: line_item.quantity,order_id: @order.is_a?(Spree::Order) ? @order.id : nil)+ # Set as targets rather than assigned, so the proposed unit is not+ # added to the line item's own fulfillment_items.+ unit.association(:line_item).target = line_item+ unit.association(:variant).target = line_item.variant+ unitendendenddiff --git a/spree/core/app/models/spree/stock/packer.rb b/spree/core/app/models/spree/stock/packer.rbindex 6726f68f2a..0bd60c091b 100644--- a/spree/core/app/models/spree/stock/packer.rb+++ b/spree/core/app/models/spree/stock/packer.rb@@ -9,9 +9,25 @@ module Spree# belong to no order yet — so anything reading the buyer through the# package (the packaging tare, the channel and company eligibility# rules) would otherwise see nothing.- def initialize(stock_location, inventory_units, splitters = [Splitter::Base], owner: nil)+ # Every candidate location's levels for the given variants in one query,+ # lowest id first as StockLocation#stock_level reads, for packing several+ # locations without a read per location.+ #+ # @param stock_locations [Array<Spree::StockLocation>]+ # @param variant_ids [Array]+ # @return [Hash{Integer => Hash{Integer => Spree::StockLevel}}] by location, then variant+ def self.stock_levels_by_location(stock_locations, variant_ids)+ Spree::StockLevel.where(stock_location_id: stock_locations.map(&:id), variant_id: variant_ids).+ order(:id).group_by(&:stock_location_id).+ transform_values { |levels| levels.group_by(&:variant_id).transform_values(&:first) }+ end++ # @param stock_levels [Hash{Integer => Spree::StockLevel}, nil] this+ # location's levels by variant id, when the caller read them already+ def initialize(stock_location, inventory_units, splitters = [Splitter::Base], owner: nil, stock_levels: nil)@stock_location = stock_location@inventory_units = inventory_units+ @stock_levels = stock_levels@splitters = splitters@owner = ownerend@@ -28,13 +44,21 @@ module Spreepackage = build_package# Group by variant_id as grouping by variant fires cached query.- inventory_units.index_by(&:variant_id).each do |variant_id, inventory_unit|- variant = Spree::Variant.find(variant_id)+ units_by_variant_id = inventory_units.index_by(&:variant_id)+ # The package's variants and their levels here, each in one query+ # rather than one per variant. Lowest id first, as StockLocation#stock_level reads.+ variants = variants_for(units_by_variant_id)+ stock_levels = @stock_levels || stock_location.stock_levels.where(variant_id: units_by_variant_id.keys).order(:id).+ group_by(&:variant_id).transform_values(&:first)++ units_by_variant_id.each do |variant_id, inventory_unit|+ variant = variants[variant_id] || Spree::Variant.find(variant_id)unit = inventory_unit.dup # Can be used by others, do not use directlyif variant.should_track_inventory?- next unless stock_location.stocks? variant+ stock_level = stock_levels[variant_id]+ next if stock_level.nil?- on_hand, backordered = stock_location.fill_status(variant, unit.quantity)+ on_hand, backordered = stock_location.fill_status(variant, unit.quantity, stock_level: stock_level)package.add(FulfillmentItem.split(unit, backordered), :backordered) if backordered.positive?package.add(FulfillmentItem.split(unit, on_hand), :on_hand) if on_hand.positive?else@@ -57,6 +81,16 @@ module Spreeprivate+ # The units' own variants when they carry them, so a cart's variants are+ # read once however many locations are packed.+ def variants_for(units_by_variant_id)+ variants = units_by_variant_id.each_with_object({}) do |(variant_id, unit), loaded|+ loaded[variant_id] = unit.variant if unit.association(:variant).loaded?+ end+ missing_ids = units_by_variant_id.keys - variants.keys+ missing_ids.empty? ? variants : variants.merge(Spree::Variant.where(id: missing_ids).index_by(&:id))+ end+def build_splittersplitter = nilsplitters.reverse_each do |klass|diff --git a/spree/core/app/models/spree/stock/quantifier.rb b/spree/core/app/models/spree/stock/quantifier.rbindex 565f4d4b65..c75d0e545d 100644--- a/spree/core/app/models/spree/stock/quantifier.rb+++ b/spree/core/app/models/spree/stock/quantifier.rb@@ -3,6 +3,22 @@ module Spreeclass Quantifierattr_reader :variant, :stock_location, :excluded_order, :provided_stock_levels+ # What a quantifier reads, preloaded for many variants at once, so+ # quantifying each of them reads memory instead of querying per variant.+ # Variants whose levels are loaded already are left as they are.+ #+ # @param variants [Enumerable<Spree::Variant>]+ # @return [void]+ def self.preload(variants)+ unloaded = variants.compact.reject { |variant| variant.association(:stock_levels).loaded? }+ return if unloaded.empty?++ ActiveRecord::Associations::Preloader.new(+ records: unloaded,+ associations: { stock_levels: [:stock_location, :active_stock_reservations] }+ ).call+ end+# @param excluded_order [Spree::Cart, Spree::Order, nil] when given,# reservations belonging to this cart/order are not counted against# availability. Used when checking a cart's own line items so thediff --git a/spree/core/app/models/spree/stock_location.rb b/spree/core/app/models/spree/stock_location.rbindex 706f969da6..e2249daecc 100644--- a/spree/core/app/models/spree/stock_location.rb+++ b/spree/core/app/models/spree/stock_location.rb@@ -182,8 +182,10 @@ module Spree## @param fulfillment [Spree::Fulfillment]# @return [Spree::StockMovement]- def allocate(variant, quantity, fulfillment)- move(variant, quantity, kind: 'allocated', cause: fulfillment)+ # @param stock_level [Spree::StockLevel, nil] the variant's level here, when+ # the caller loaded it already+ def allocate(variant, quantity, fulfillment, stock_level: nil)+ move(variant, quantity, kind: 'allocated', cause: fulfillment, stock_level: stock_level)end# Withdraws a promise — a canceled, relocated or shrunk fulfillment.@@ -199,21 +201,26 @@ module Spree# a negative quantity writes stock off.## @param reason [String] audit text, required+ # @param stock_level [Spree::StockLevel, nil] the variant's level here, when+ # the caller holds it already (locked, say), so the movement updates that+ # instance rather than one looked up again# @return [Spree::StockMovement]- def adjust(variant, quantity, reason:)- move(variant, quantity, kind: 'adjusted', reason: reason)+ def adjust(variant, quantity, reason:, stock_level: nil)+ move(variant, quantity, kind: 'adjusted', reason: reason, stock_level: stock_level)end- def move(variant, quantity, kind:, cause: nil, reason: nil, force: false, unit_cost: nil)- stock_level = stock_level_or_create(variant)+ def move(variant, quantity, kind:, cause: nil, reason: nil, force: false, unit_cost: nil, stock_level: nil)+ stock_level ||= stock_level_or_create(variant)attributes = { quantity: quantity, kind: kind, reason: reason, unit_cost: unit_cost,**cause_attributes(cause) }stock_level.stock_movements.create!(attributes) { |movement| movement.force = force }end- def fill_status(variant, quantity)- if item = stock_level_or_create(variant)+ # @param stock_level [Spree::StockLevel, nil] the variant's level here, when+ # the caller loaded it already+ def fill_status(variant, quantity, stock_level: nil)+ if item = stock_level || stock_level_or_create(variant)if item.available_count >= quantityon_hand = quantitybackordered = 0diff --git a/spree/core/app/models/spree/variant.rb b/spree/core/app/models/spree/variant.rbindex 81f841d4d6..0702357dcd 100644--- a/spree/core/app/models/spree/variant.rb+++ b/spree/core/app/models/spree/variant.rb@@ -393,7 +393,11 @@ module Spree@tax_category ||= if self[:tax_category_id].nil?product.tax_categoryelse- Spree::TaxCategory.find_by(id: self[:tax_category_id]) || product.tax_category+ # The association, so a page of variants loads its+ # categories together; one deleted since it was+ # assigned no longer counts.+ own = super+ own.nil? || own.destroyed? || own.try(:deleted?) ? product.tax_category : ownendend@@ -466,7 +470,7 @@ module Spree# Returns the options text of the variant.# @return [String] the options text of the variantdef options_text- @options_text ||= if option_values.loaded?+ @options_text ||= if read_association_in_memory?(:option_values)option_values.sort_by do |ov|ov.option_type.positionend.map { |ov| "#{ov.option_type.label}: #{ov.label}" }.to_sentence(words_connector: ', ', two_words_connector: ', ')@@ -663,7 +667,7 @@ module Spreedef price_in(currency)currency = currency&.upcase- price = if read_association_in_memory?(:prices) && prices.any?+ price = if read_association_in_memory?(:prices)prices.detect { |p| p.currency == currency && p.price_list_id.nil? }elseprices.base_prices.find_by(currency: currency)@@ -864,7 +868,12 @@ module Spree# @return [void]def set_stock(count_on_hand, backorderable = nil, stock_location = nil)stock_location ||= default_stock_location- stock_level = stock_levels.find_or_initialize_by(stock_location: stock_location)+ stock_level = if stock_levels.loaded?+ stock_levels.detect { |level| level.stock_location_id == stock_location.id } ||+ stock_levels.build(stock_location: stock_location)+ else+ stock_levels.find_or_initialize_by(stock_location: stock_location)+ end# `unless nil?`, not `if present?` — a caller passing false means it.stock_level.backorderable = backorderable unless backorderable.nil?@@ -880,7 +889,10 @@ module Spreedelta = count_on_hand.to_i - stock_level.count_on_hand.to_ireturn if delta.zero?- stock_location.adjust(self, delta, reason: Spree::StockMovement.default_adjustment_reason)+ # Through the level already in hand rather than StockLocation#adjust,+ # which would look the same row up again.+ stock_level.stock_movements.create!(quantity: delta, kind: 'adjusted',+ reason: Spree::StockMovement.default_adjustment_reason)enddef default_stock_location@@ -967,6 +979,11 @@ module Spree# Variants carry no store of their own; the product owns it.# @return [Spree::Store, nil]def preference_store+ # The request's store instance when it is this one, so a page of+ # variants reads its settings from one record rather than a copy each.+ current_store = Spree::Current.store+ return current_store if current_store && product&.store_id == current_store.id+product&.storeend@@ -1188,7 +1205,7 @@ module Spreeenddef set_cost_currency- self.cost_currency = Spree::Store.default.default_currency if cost_currency.blank?+ self.cost_currency = (product&.store || Spree::Store.default).default_currency if cost_currency.blank?end# Only the product's own store's locations — a new variant must not growdiff --git a/spree/core/app/services/spree/carts/price_items.rb b/spree/core/app/services/spree/carts/price_items.rbindex f0257cddf6..391cd351ec 100644--- a/spree/core/app/services/spree/carts/price_items.rb+++ b/spree/core/app/services/spree/carts/price_items.rb@@ -31,6 +31,9 @@ module Spreeitems = items.reject(&:manual_price?)return success([]) if items.empty?+ # One read of every line's prices rather than one per variant.+ ActiveRecord::Associations::Preloader.new(records: items.filter_map(&:variant).uniq, associations: :prices).call+resolutions = items.filter_map do |line_item|price = resolve_price(line_item, cart)next if price.blank?diff --git a/spree/core/app/services/spree/carts/split_by_seller.rb b/spree/core/app/services/spree/carts/split_by_seller.rbindex 062fc2b876..4c9ec04f7c 100644--- a/spree/core/app/services/spree/carts/split_by_seller.rb+++ b/spree/core/app/services/spree/carts/split_by_seller.rb@@ -270,8 +270,10 @@ module Spree# describe what they held before the split, and the new one holds nothing# at all.def restate_freight_summaries(*fulfillments)+ selected_rates = Spree::DeliveryRate.where(fulfillment_id: fulfillments.map(&:id), selected: true).order(:cost).to_a+fulfillments.each do |fulfillment|- rate = fulfillment.reload.selected_delivery_rate+ rate = selected_rates.find { |selected_rate| selected_rate.fulfillment_id == fulfillment.id }next if rate.nil?metadata = rate.metadatadiff --git a/spree/core/app/services/spree/catalogs/set_product_terms.rb b/spree/core/app/services/spree/catalogs/set_product_terms.rbindex 1ce2f41594..1abedd7235 100644--- a/spree/core/app/services/spree/catalogs/set_product_terms.rb+++ b/spree/core/app/services/spree/catalogs/set_product_terms.rb@@ -50,7 +50,8 @@ module Spreedef apply(catalog, product, values)minimum = normalize(values[:minimum_order_quantity])multiple = normalize(values[:order_multiple])- variant_ids = product.variants.ids+ variants = product.variants.to_a+ variant_ids = variants.map(&:id)# Both blank is a deletion: the product falls back to the catalog's# own default, which is what an empty pair means on the row.@@ -59,8 +60,12 @@ module Spreereturn nilend- variant_ids.each do |variant_id|- rule = catalog.quantity_rules.find_or_initialize_by(variant_id: variant_id)+ existing_rules = catalog.quantity_rules.where(variant_id: variant_ids).index_by(&:variant_id)++ variants.each do |variant|+ rule = existing_rules[variant.id] || catalog.quantity_rules.build+ # The loaded variant, so the store check reads its product in memory.+ rule.variant = variantrule.assign_attributes(minimum_order_quantity: minimum, order_multiple: multiple)return rule unless rule.saveenddiff --git a/spree/core/app/services/spree/collections/add_products.rb b/spree/core/app/services/spree/collections/add_products.rbindex 5dc906a3dd..3eed9a8df9 100644--- a/spree/core/app/services/spree/collections/add_products.rb+++ b/spree/core/app/services/spree/collections/add_products.rb@@ -22,7 +22,7 @@ module Spreedef group_class = Spree::Collectiondef refresh_group_counters(collection_ids)- collection_ids.each { |id| Spree::Collection.reset_counters(id, :product_collections) }+ Spree::Collection.reset_counters_in_bulk(collection_ids, :product_collections)endendenddiff --git a/spree/core/app/services/spree/collections/regenerate_products.rb b/spree/core/app/services/spree/collections/regenerate_products.rbindex 4bb4bd8876..245b85f783 100644--- a/spree/core/app/services/spree/collections/regenerate_products.rb+++ b/spree/core/app/services/spree/collections/regenerate_products.rb@@ -44,9 +44,7 @@ module Spree# counter caches — bulk insert/delete bypass the ProductCollection counter_cache callbacksSpree::Collection.reset_counters(collection.id, :product_collections)- all_product_ids = (previous_products_ids + product_ids_to_insert).uniq- existing_product_ids = Spree::Product.where(id: all_product_ids).pluck(:id)- existing_product_ids.each { |id| Spree::Product.reset_counters(id, :product_collections) }+ Spree::Product.reset_counters_in_bulk(previous_products_ids + product_ids_to_insert, :product_collections)success(collection)enddiff --git a/spree/core/app/services/spree/collections/remove_products.rb b/spree/core/app/services/spree/collections/remove_products.rbindex 554188032e..649a453684 100644--- a/spree/core/app/services/spree/collections/remove_products.rb+++ b/spree/core/app/services/spree/collections/remove_products.rb@@ -22,7 +22,7 @@ module Spreedef group_class = Spree::Collectiondef refresh_group_counters(collection_ids)- collection_ids.each { |id| Spree::Collection.reset_counters(id, :product_collections) }+ Spree::Collection.reset_counters_in_bulk(collection_ids, :product_collections)endendenddiff --git a/spree/core/app/services/spree/order_groups/allocate_payments.rb b/spree/core/app/services/spree/order_groups/allocate_payments.rbindex cf768be393..20327ae9ae 100644--- a/spree/core/app/services/spree/order_groups/allocate_payments.rb+++ b/spree/core/app/services/spree/order_groups/allocate_payments.rb@@ -82,14 +82,16 @@ module Spree)captured = payment.completed?+ allocated_order_ids = Spree::PaymentSplit.where(payment_id: payment.id, order_id: orders.map(&:id)).pluck(:order_id).to_set+orders.each_with_index do |order, index|- split = Spree::PaymentSplit.find_or_initialize_by(payment_id: payment.id, order_id: order.id)- next if split.persisted?+ next if allocated_order_ids.include?(order.id)share = Spree::Money::Rounding.from_minor_units(shares[index], currency)- # So re-summing its payment total on save reuses this instance rather- # than loading a second copy of the row per share.- split.order = order+ # The payment and order are handed over rather than looked up, so+ # validating and re-summing each share reuses these instances+ # instead of loading a copy per share.+ split = Spree::PaymentSplit.new(payment: payment, order: order)split.currency = payment.currencysplit.authorized_amount = sharesplit.captured_amount = captured ? share : 0diff --git a/spree/core/app/services/spree/product_memberships.rb b/spree/core/app/services/spree/product_memberships.rbindex 3fbc6572b2..394b741e08 100644--- a/spree/core/app/services/spree/product_memberships.rb+++ b/spree/core/app/services/spree/product_memberships.rb@@ -7,9 +7,15 @@ module Spreeprivatedef add_memberships(groups, products)- rows = groups.pluck(:id).flat_map do |group_id|- position = membership_class.where(group_key => group_id).count- products.pluck(:id).map do |product_id|+ group_ids = groups.pluck(:id)+ product_ids = products.pluck(:id)+ # Every group's current size in one grouped count, rather than a probe+ # per group for the position to append at.+ sizes = membership_class.where(group_key => group_ids).group(group_key).count++ rows = group_ids.flat_map do |group_id|+ position = sizes.fetch(group_id, 0)+ product_ids.map do |product_id|{group_key => group_id, product_id: product_id, position: (position += 1),created_at: Time.current, updated_at: Time.current@@ -18,7 +24,7 @@ module Spreeendmembership_class.insert_all(rows)- refresh_after_membership_change(groups.pluck(:id), products)+ refresh_after_membership_change(group_ids, products)end# Removes the products, then re-packs the remaining positions from 1.@@ -28,8 +34,12 @@ module SpreeApplicationRecord.transaction domembership_class.where(group_key => group_ids, product_id: products.pluck(:id)).delete_all+ # What survived the delete, for every group in one read.+ remaining = membership_class.where(group_key => group_ids).order(:position).pluck(group_key, :product_id).+ group_by(&:first).transform_values { |pairs| pairs.map(&:last) }+rows = group_ids.flat_map do |group_id|- membership_class.where(group_key => group_id).order(:position).pluck(:product_id).map.with_index(1) do |product_id, position|+ remaining.fetch(group_id, []).map.with_index(1) do |product_id, position|{group_key => group_id, product_id: product_id, position: position,created_at: Time.current, updated_at: Time.current@@ -46,7 +56,7 @@ module Spree# search index are refreshed here.def refresh_after_membership_change(group_ids, products)product_ids = products.pluck(:id)- product_ids.each { |id| Spree::Product.reset_counters(id, membership_class.model_name.element.pluralize.to_sym) }+ Spree::Product.reset_counters_in_bulk(product_ids, membership_class.model_name.element.pluralize.to_sym)refresh_group_counters(group_ids)Spree::Product.where(id: product_ids).touch_alldiff --git a/spree/core/app/services/spree/stock_levels/correct.rb b/spree/core/app/services/spree/stock_levels/correct.rbindex d636e1884d..dc45a08eb2 100644--- a/spree/core/app/services/spree/stock_levels/correct.rb+++ b/spree/core/app/services/spree/stock_levels/correct.rb@@ -32,6 +32,11 @@ module Spreereturn failure(stock_level, I18n.t('spree.stock_level.errors.count_on_hand_not_an_integer')) if count_on_hand.present? && target.nil?return failure(stock_level, I18n.t('spree.stock_level.errors.adjustment_not_an_integer')) if adjustment.present? && delta.nil?+ # Read before locking, which reloads the level and with it drops the+ # variant and location a bulk caller loaded for every row at once.+ stock_location = stock_level.stock_location+ variant = stock_level.variant+# Locked around the read: the delta is worked out from the count this# caller first saw, so two admins correcting the same level at once# would otherwise each measure against a shelf the other had already@@ -42,12 +47,14 @@ module Spree# Through the location's own mover rather than building a movement# by hand: it types the row as an adjustment and records the reason,- # so a correction reads like every other change in the history.- stock_level.stock_location.adjust(stock_level.variant, move,- reason: Spree::StockMovement.adjustment_reason_text(reason))+ # so a correction reads like every other change in the history. The+ # movement writes through this locked instance, which leaves it+ # current without another reload.+ stock_location.adjust(variant, move, reason: Spree::StockMovement.adjustment_reason_text(reason),+ stock_level: stock_level)end- success(stock_level.reload)+ success(stock_level)endprivatediff --git a/spree/core/app/services/spree/stock_reservations/reserve.rb b/spree/core/app/services/spree/stock_reservations/reserve.rbindex 38671b93fd..4c328e6fd6 100644--- a/spree/core/app/services/spree/stock_reservations/reserve.rb+++ b/spree/core/app/services/spree/stock_reservations/reserve.rb@@ -69,7 +69,7 @@ module Spreeprivatedef build_targets(cart)- cart.line_items.includes(variant: { stock_levels: :stock_location }).filter_map do |line_item|+ cart.line_items.includes(variant: [:product, { stock_levels: :stock_location }]).filter_map do |line_item|variant = line_item.variantnext unless variant&.should_track_inventory?diff --git a/spree/core/app/services/spree/tags/bulk_add.rb b/spree/core/app/services/spree/tags/bulk_add.rbindex 6212437429..0484e0fcff 100644--- a/spree/core/app/services/spree/tags/bulk_add.rb+++ b/spree/core/app/services/spree/tags/bulk_add.rb@@ -10,8 +10,15 @@ module Spree# @param context [String] default: 'tags'# @return [Spree::ServiceModule::Base::Result]def call(tag_names: [], records: [], context: 'tags')- tags = tag_names.map do |tag_name|- ActsAsTaggableOn::Tag.find_or_create_by(name: tag_name.strip)+ names = tag_names.map(&:strip).reject(&:blank?).uniq+ return if names.empty?++ # One read for the names that already exist, then a create only for+ # the ones that do not. `find_or_create_by` per name is a query per+ # tag, which a bulk tagging pays on every name in the payload.+ existing = ActsAsTaggableOn::Tag.where(name: names).index_by(&:name)+ tags = names.map do |name|+ existing[name] || ActsAsTaggableOn::Tag.create_or_find_by(name: name)endrecord_class = records.first.classdiff --git a/spree/core/app/workflows/spree/carts/complete.rb b/spree/core/app/workflows/spree/carts/complete.rbindex 7e796e22d6..e4dcb919ac 100644--- a/spree/core/app/workflows/spree/carts/complete.rb+++ b/spree/core/app/workflows/spree/carts/complete.rb@@ -422,10 +422,21 @@ module Spree# provider a question whose answer is discarded — and for an external# engine that is a billable remote call per line item, inside the# completion lock.+ #+ # Each copy is handed the cart line's variant, loaded once for every+ # line, so validating the copies reads its product, stock and tax+ # category for all of them at once rather than per line.def copy_line_items!(cart, order)- cart.line_items.reload.index_with do |cart_line_item|+ cart_line_items = cart.line_items.reload.to_a+ ActiveRecord::Associations::Preloader.new(+ records: cart_line_items,+ associations: { variant: [{ product: %i[store product_publications tax_category] }, { stock_levels: %i[stock_location active_stock_reservations] }] }+ ).call++ cart_line_items.index_with do |cart_line_item|attributes = cart_line_item.attributes.except('id', 'cart_id', 'created_at', 'updated_at')line_item = order.line_items.new(attributes.merge('order_id' => order.id))+ line_item.variant = cart_line_item.variant if cart_line_item.variantline_item.skip_tax_estimation = trueline_item.save!line_item@@ -434,9 +445,12 @@ module Spree# @return [Hash{Integer => Integer}] cart fulfillment id → order fulfillment iddef copy_fulfillments!(cart, order, line_item_map)- line_item_id_map = line_item_map.transform_keys(&:id).transform_values(&:id)+ line_items_by_cart_line_item_id = line_item_map.transform_keys(&:id)++ cart_fulfillments = cart.fulfillments.reload.to_a+ ActiveRecord::Associations::Preloader.new(records: cart_fulfillments, associations: %i[selected_delivery_rate fulfillment_items]).call- cart.fulfillments.reload.each_with_object({}) do |cart_fulfillment, map|+ cart_fulfillments.each_with_object({}) do |cart_fulfillment, map|attributes = cart_fulfillment.attributes.except('id', 'cart_id', 'number', 'tracking', 'created_at', 'updated_at')fulfillment = order.fulfillments.create!(attributes.merge('order_id' => order.id, 'address_id' => order.ship_address_id))@@ -447,13 +461,15 @@ module Spreecart_fulfillment.fulfillment_items.each do |item|item_attributes = item.attributes.except('id', 'created_at', 'updated_at')- fulfillment.fulfillment_items.create!(- item_attributes.merge(- 'fulfillment_id' => fulfillment.id,- 'order_id' => order.id,- 'line_item_id' => line_item_id_map.fetch(item.line_item_id)- )+ line_item = line_items_by_cart_line_item_id.fetch(item.line_item_id)+ fulfillment_item = fulfillment.fulfillment_items.new(+ item_attributes.merge('fulfillment_id' => fulfillment.id, 'order_id' => order.id))+ # Objects rather than ids, so validating the copy does not look+ # each of them up again.+ fulfillment_item.line_item = line_item+ fulfillment_item.variant = line_item.variant if line_item.variant_id == item.variant_id+ fulfillment_item.save!endmap[cart_fulfillment.id] = fulfillment.iddiff --git a/spree/core/app/workflows/spree/carts/upsert_items.rb b/spree/core/app/workflows/spree/carts/upsert_items.rbindex 7df201e041..940c731b5a 100644--- a/spree/core/app/workflows/spree/carts/upsert_items.rb+++ b/spree/core/app/workflows/spree/carts/upsert_items.rb@@ -90,9 +90,10 @@ module Spree# applied.def resolve_itemsstore = cart.store || Spree::Current.store+ requested_items = Array(items).map { |item_params| item_params.to_h.deep_symbolize_keys }+ prefetch_added_variants(store, requested_items)- @resolved_items = Array(items).filter_map do |item_params|- item_params = item_params.to_h.deep_symbolize_keys+ @resolved_items = requested_items.filter_map do |item_params|quantity = (item_params[:quantity] || 1).to_ivariant = resolve_variant(store, item_params[:variant_id], removing: quantity <= 0)next if variant.nil?@@ -357,18 +358,44 @@ module Spreecart.line_items.detect { |line_item| line_item.variant_id.to_s == variant_id.to_s }&.variant ||Spree::Variant.with_deleted.find_by_param(variant_id)else- variant = store.variants.find_by_param(variant_id)+ variant = @added_variants[variant_param_id(variant_id)]# +orderable_variants+ narrows what a request may add. A line already# in the cart is a quantity edit and resolves as before — checkout# judges it — so one stale line cannot fail a whole cart's update.if variant && orderable_variants && cart.line_items.none? { |line_item| line_item.variant_id == variant.id }- variant = orderable_variants.find_by(id: variant.id)+ variant = nil unless @orderable_added_variant_ids.include?(variant.id)endvariant || raise(ActiveRecord::RecordNotFound.new("Variant '#{variant_id}' not found in this store", 'Spree::Variant', 'id', variant_id))endend+ # Every variant the batch adds, looked up through the store in one+ # query rather than one per item, and which of them the buyer may order.+ def prefetch_added_variants(store, requested_items)+ ids = requested_items.filter_map do |item_params|+ variant_param_id(item_params[:variant_id]) if (item_params[:quantity] || 1).to_i.positive?+ end++ @added_variants = ids.empty? ? {} : store.variants.where(id: ids.uniq).preload_associations_lazily.index_by(&:id)+ Spree::Stock::Quantifier.preload(@added_variants.values)+ @orderable_added_variant_ids = if orderable_variants && @added_variants.any?+ orderable_variants.where(id: @added_variants.keys).pluck(:id).to_set+ else+ Set.new+ end+ end++ # The id a variant param names: a prefixed id, or a bare integer id+ # kept for backwards compatibility.+ #+ # @return [Integer, nil]+ def variant_param_id(param)+ return nil if param.blank?++ Spree::Variant.decode_prefixed_id(param) || (param.to_s.to_i if param.to_s.match?(/\A\d+\z/))+ end+def warn(index, item, code, message)warnings << Spree::Carts::ItemWarning.new(item_index: index, variant: item.variant, code: code, message: message)enddiff --git a/spree/core/app/workflows/spree/customers/anonymize.rb b/spree/core/app/workflows/spree/customers/anonymize.rbindex 3922324e1f..bbdbd4c847 100644--- a/spree/core/app/workflows/spree/customers/anonymize.rb+++ b/spree/core/app/workflows/spree/customers/anonymize.rb@@ -318,7 +318,7 @@ module Spree}owner_ids.each do |owner_type, ids|- Spree::ShippingLabel.where(owner_type: owner_type, owner_id: ids).find_each do |label|+ Spree::ShippingLabel.where(owner_type: owner_type, owner_id: ids).with_attached_file.find_each do |label|purge_after_commit(label.file)endend@@ -408,7 +408,7 @@ module Spree# complete copy of everything being erased, sitting behind a link that# stays live for days. Rewriting the row's email would leave the# payload itself untouched and still fetchable.- requests.each do |data_request|+ requests.with_attached_export_file.each do |data_request|purge_after_commit(data_request.export_file)enddiff --git a/spree/core/app/workflows/spree/imports/start_mapping.rb b/spree/core/app/workflows/spree/imports/start_mapping.rbindex 993d6dc153..4ff9d221d8 100644--- a/spree/core/app/workflows/spree/imports/start_mapping.rb+++ b/spree/core/app/workflows/spree/imports/start_mapping.rb@@ -33,8 +33,10 @@ module Spree# TODO: seed from the previous import of the same type, so a merchant# doesn't map the same columns twice.def build_mappings+ existing = import.mappings.index_by(&:schema_field)+import.schema_fields.each do |schema_field|- mapping = import.mappings.find_or_create_by!(schema_field: schema_field[:name])+ mapping = existing[schema_field[:name]] || import.mappings.build(schema_field: schema_field[:name])mapping.try_to_auto_assign_file_column(import.csv_headers)mapping.save!enddiff --git a/spree/core/app/workflows/spree/orders/complete.rb b/spree/core/app/workflows/spree/orders/complete.rbindex 81199de255..5ad4612878 100644--- a/spree/core/app/workflows/spree/orders/complete.rb+++ b/spree/core/app/workflows/spree/orders/complete.rb@@ -140,11 +140,16 @@ module Spreeenddef allocate_fulfillment_stock(fulfillment)- fulfillment.manifest.each do |item|- next unless item.variant.track_inventory?- next unless item.quantity.positive?+ items = fulfillment.manifest.select { |item| item.variant.track_inventory? && item.quantity.positive? }+ return if items.empty?- fulfillment.stock_location.allocate(item.variant, item.quantity, fulfillment)+ stock_location = fulfillment.stock_location+ # Lowest id first, as StockLocation#stock_level reads.+ stock_levels = stock_location.stock_levels.where(variant_id: items.map { |item| item.variant.id }).order(:id).+ group_by(&:variant_id).transform_values(&:first)++ items.each do |item|+ stock_location.allocate(item.variant, item.quantity, fulfillment, stock_level: stock_levels[item.variant.id])endend@@ -240,7 +245,9 @@ module Spree# wildcard subscribers dedupe on the metadata marker.def publish_order_placedmetadata = { notify_customer: order.notify_customer }- payload = order.event_payload+ # The payload renders every line and parcel, read back through a lazily+ # preloading query so each association loads once rather than per row.+ payload = Spree::Order.where(id: order.id).preload_associations_lazily.take!.event_payloadorder.publish_event('order.placed', payload, metadata)order.publish_event('order.completed', payload, metadata.merge(deprecated_alias_of: 'order.placed'))enddiff --git a/spree/core/app/workflows/spree/stock_levels/bulk_upsert.rb b/spree/core/app/workflows/spree/stock_levels/bulk_upsert.rbindex 7d48e42587..df553a754b 100644--- a/spree/core/app/workflows/spree/stock_levels/bulk_upsert.rb+++ b/spree/core/app/workflows/spree/stock_levels/bulk_upsert.rb@@ -35,9 +35,10 @@ module Spree# means the later line, not the sum of both.deduped = keyed.reverse.uniq { |row| [row[:variant_id].to_s, row[:stock_location_id].to_s] }.reverse+ existing = existing_stock_levels(deduped)count = 0Spree::StockLevel.transaction do- deduped.each { |row| count += 1 if apply_row(row) }+ deduped.each { |row| count += 1 if apply_row(row, existing) }endsuccess(stock_level_count: count)@@ -52,27 +53,41 @@ module Spree# happens to name — including another merchant's. One caller decoding# ids correctly does not make the workflow safe for the next one.def resolve(rows)- variants = scoped_ids(store_variants, Spree::Variant, rows.map { |row| row[:variant_id] })- locations = scoped_ids(store_stock_locations, Spree::StockLocation,- rows.map { |row| row[:stock_location_id] })+ @variants = scoped_records(store_variants, Spree::Variant, rows.map { |row| row[:variant_id] })+ @stock_locations = scoped_records(store_stock_locations, Spree::StockLocation,+ rows.map { |row| row[:stock_location_id] })+ @variants_by_id = @variants.values.index_by(&:id)+ @stock_locations_by_id = @stock_locations.values.index_by(&:id)rows.map do |row|row.merge(- variant_id: variants[row[:variant_id]],- stock_location_id: locations[row[:stock_location_id]]+ variant_id: @variants[row[:variant_id]]&.id,+ stock_location_id: @stock_locations[row[:stock_location_id]]&.id).compactendend# Accepts a prefixed id or a raw one: the Admin API speaks prefixed,# while the importer and seeds hold records they already loaded.- def scoped_ids(scope, model, values)+ #+ # @return [Hash{Object => ActiveRecord::Base}] each value as given => this store's record+ def scoped_records(scope, model, values)decoded = values.compact.uniq.index_with do |value|model.decode_own_prefixed_id(value) || valueend- found = scope.where(id: decoded.values.compact).pluck(:id).map(&:to_s).to_set+ found = scope.where(id: decoded.values.compact).index_by { |record| record.id.to_s }++ decoded.transform_values { |id| found[id.to_s] if id }.compact+ end- decoded.transform_values { |id| id if id && found.include?(id.to_s) }+ # Every shelf the feed names, in one query rather than one per row.+ # Lowest id wins, as an unordered lookup per pair would return.+ def existing_stock_levels(rows)+ Spree::StockLevel.+ where(variant_id: rows.map { |row| row[:variant_id] }.uniq,+ stock_location_id: rows.map { |row| row[:stock_location_id] }.uniq).+ order(id: :desc).+ index_by { |stock_level| [stock_level.variant_id.to_s, stock_level.stock_location_id.to_s] }enddef store_variants@@ -83,10 +98,10 @@ module Spree@store.stock_locationsend- def apply_row(row)- stock_level = Spree::StockLevel.find_or_initialize_by(- variant_id: row[:variant_id], stock_location_id: row[:stock_location_id]- )+ def apply_row(row, existing)+ stock_level = existing[[row[:variant_id].to_s, row[:stock_location_id].to_s]] || Spree::StockLevel.new+ stock_level.variant = @variants_by_id.fetch(row[:variant_id])+ stock_level.stock_location = @stock_locations_by_id.fetch(row[:stock_location_id])stock_level.backorderable = row[:backorderable] if row.key?(:backorderable)stock_level.save! if stock_level.new_record? || stock_level.changed?diff --git a/spree/core/app/workflows/spree/variants/nested_attributes.rb b/spree/core/app/workflows/spree/variants/nested_attributes.rbindex e4b3a71940..7093db3565 100644--- a/spree/core/app/workflows/spree/variants/nested_attributes.rb+++ b/spree/core/app/workflows/spree/variants/nested_attributes.rb@@ -48,6 +48,8 @@ module Spreereturn if prices_params.nil?currencies = []+ # Loaded once so each currency's write finds its row in memory.+ variant.prices.load if variant.persisted?Array(prices_params).each do |raw|price = raw.to_h.with_indifferent_access@@ -59,7 +61,8 @@ module Spreereturn unless variant.persisted?- variant.prices.base_prices.where.not(currency: currencies).destroy_all+ stale = variant.prices.select { |price| price.persisted? && price.price_list_id.nil? && currencies.exclude?(price.currency) }+ variant.prices.destroy(*stale) if stale.any?end# @param variant [Spree::Variant]@@ -68,22 +71,44 @@ module Spreedef apply_stock_levels(variant, stock_levels_params)return if stock_levels_params.blank?- Array(stock_levels_params).each do |raw|- row = raw.to_h.with_indifferent_access- next if row[:stock_location_id].blank?+ rows = Array(stock_levels_params).map { |raw| raw.to_h.with_indifferent_access }+ rows.reject! { |row| row[:stock_location_id].blank? }+ return if rows.empty?- # Scoped through the product's own store: Spree::StockLocation- # lookup is global, so an id belonging to another store would- # otherwise put this variant's stock in that store's warehouse.- location = stock_location_for(variant, row[:stock_location_id])+ locations = stock_locations_by_param(variant, rows.map { |row| row[:stock_location_id] })+ # Loaded once so each row's write finds its level in memory.+ variant.stock_levels.load if variant.persisted?++ rows.each do |row|+ location = locations[row[:stock_location_id].to_s]next if location.nil?variant.set_stock(row[:count_on_hand], row[:backorderable], location)end++ # A count change locks and reloads its level, so the locations the+ # response's stock figures read are loaded for every level afterwards.+ ActiveRecord::Associations::Preloader.new(records: variant.stock_levels.to_a, associations: :stock_location).callend- def stock_location_for(variant, param)- variant.product&.store&.stock_locations&.find_by_param(param)+ # Resolves every stock location the payload names in one read. Scoped+ # through the product's own store: Spree::StockLocation lookup is+ # global, so an id belonging to another store would otherwise put this+ # variant's stock in that store's warehouse. A store keeps few+ # locations, so loading them beats a lookup per row.+ #+ # @return [Hash{String => Spree::StockLocation}] keyed by both the+ # prefixed id and the raw id, which is what the payload may carry+ def stock_locations_by_param(variant, params)+ scope = variant.product&.store&.stock_locations+ return {} if scope.nil? || params.empty?++ wanted = params.map(&:to_s).uniq+ scope.each_with_object({}) do |location, memo|+ [location.prefixed_id.to_s, location.id.to_s].each do |key|+ memo[key] = location if wanted.include?(key)+ end+ endendendenddiff --git a/spree/core/lib/generators/spree/dummy/templates/rails/test.rb b/spree/core/lib/generators/spree/dummy/templates/rails/test.rbindex 7ae3d63ccd..7ae8215228 100644--- a/spree/core/lib/generators/spree/dummy/templates/rails/test.rb+++ b/spree/core/lib/generators/spree/dummy/templates/rails/test.rb@@ -77,4 +77,91 @@ Dummy::Application.configure doconfig.active_record.encryption.key_derivation_salt = 'spree-test-key-derivation-salt'routes.default_url_options = { host: 'localhost', port: 3000 }++ # An N+1 inside an API request fails the example instead of being logged,+ # so a serializer or model method that queries once per record cannot land+ # unnoticed. Wrap a deliberately repeated lookup in Prosopite.pause.+ if defined?(Prosopite)+ # Hooked from Spree's own test app rather than declared in the+ # controller, so a host application that bundles Prosopite for itself is+ # never opted into Spree's guard. Prepended so the lookups made by the+ # authentication and parent callbacks are scanned too.+ config.to_prepare do+ if defined?(Spree::Api::V3::BaseController)+ Spree::Api::V3::BaseController.prepend_around_action { |_controller, action| Prosopite.scan(&action) }+ end+ end++ Prosopite.rails_logger = true+ # Raise only under RSpec, where a report belongs to the example that+ # caused it. The same dummy app serves a real server for the SDK and+ # browser suites, and there a raise becomes a 500 the client cannot act+ # on — those runs log instead. SPREE_N_PLUS_ONE_GUARD overrides either+ # way ('raise', 'log').+ running_specs = ENV.key?('TEST_ENV_NUMBER') || $PROGRAM_NAME.end_with?('rspec')+ guard_mode = ENV.fetch('SPREE_N_PLUS_ONE_GUARD') { running_specs ? 'raise' : 'log' }+ Prosopite.raise = guard_mode == 'raise'++ # The Rails cleaner keeps only the host app's frames, which hides the+ # serializer or model line that issued the repeated query.+ Prosopite.backtrace_cleaner = ActiveSupport::BacktraceCleaner.new.tap do |cleaner|+ cleaner.add_silencer { |line| line.include?('/gems/') || line.include?('/rspec') }+ end++ # The guard covers what a request reads. Writes run their callbacks one+ # record at a time by design — a `dependent: :destroy` cascade, a number+ # or uniqueness probe per created row — so statements issued under a+ # save or destroy are not reported.+ Prosopite.allow_stack_paths = [+ %r{active_record/persistence\.rb:\d+:in '[^']*#(save|save!|destroy|destroy!|touch)'},+ # A touch is a write too, and Spree overrides `touch` in its own+ # concern, so the frame is Spree's rather than Active Record's.+ %r{active_record/timestamp\.rb:\d+:in '[^']*#touch'},+ %r{/spree/publishable\.rb:\d+:in '[^']*#touch'},+ %r{/paranoia\.rb:\d+:in '[^']*destroy'},+ # Validations run before the persistence frame starts, so a uniqueness+ # or ladder probe on the row being saved needs its own entry.+ %r{active_record/validations\.rb:\d+:in '[^']*#(save|save!)'},+ # after_commit callbacks are the same per-record write work, deferred+ # until the transaction that wrote the rows ends.+ %r{active_record/transactions\.rb:\d+:in '[^']*#committed!'},+ # Locking a row re-reads it, once per row the caller is about to write.+ %r{active_record/locking/pessimistic\.rb:\d+:in '[^']*#lock!'},+ # A product payload writes each nested variant and media row through its+ # own workflow — SKU probe, option lookup, blob resolution — once per row.+ %r{/spree/products/nested_attributes\.rb:\d+:in '[^']*#(write_variant|apply_media)'},+ # A settlement is swept per currency, and each sweep writes its own payout.+ %r{/spree/seller_payouts/sweep\.rb:\d+:in '[^']*#collect_transfers'},+ # A split checkout re-sums and then places each seller's order it just+ # wrote, one workflow run per order.+ %r{/spree/carts/split_by_seller\.rb:\d+:in '[^']*#resum_totals'},+ %r{/spree/orders/complete\.rb:\d+:in '[^']*#place_sibling'},+ # Each requested line is priced, stock-checked, saved and taxed as it+ # is written.+ %r{/spree/carts/upsert_items\.rb:\d+:in '[^']*#apply_item'}+ ]++ # A purchase's billing and shipping addresses, and a transfer's two+ # warehouses, are two links into one table read through the same code,+ # so one record reads as a repeat. Lists still load each link in one+ # query for the whole page.+ Prosopite.ignore_queries = [+ /FROM [`"]?(spree_addresses|spree_stock_locations)[`"]? WHERE (.+ AND )?[`"]?\1[`"]?\.[`"]?id[`"]? (=|IN)/+ ]++ # Prosopite fingerprints through pg_query, which rejects SQLite's `?`+ # binds, so SQLite borrows the text normalizer written for MySQL. That+ # normalizer blanks anything double-quoted, which is a string literal in+ # MySQL but an identifier in SQLite — left alone it erases every table+ # name, collapsing unrelated queries into one fingerprint and reporting+ # a single record's distinct associations as an N+1. Re-quoting+ # identifiers as backticks first keeps table names in the fingerprint.+ Prosopite.singleton_class.prepend(Module.new do+ def fingerprint(query)+ return super unless ActiveRecord::Base.connection_db_config.adapter == 'sqlite3'++ mysql_fingerprint(query.gsub(/"([^"]*)"/) { "`#{Regexp.last_match(1)}`" })+ end+ end)+ endenddiff --git a/spree/core/lib/spree/checkout/default_requirements.rb b/spree/core/lib/spree/checkout/default_requirements.rbindex 6c46e036b5..e389ed0ea7 100644--- a/spree/core/lib/spree/checkout/default_requirements.rb+++ b/spree/core/lib/spree/checkout/default_requirements.rb@@ -107,7 +107,11 @@ module Spreeenddef completion_line_items- @completion_line_items ||= @cart.line_items.includes(variant: :product).to_a+ # Everything the discontinuation and stock checks read, for every line+ # at once.+ @completion_line_items ||= @cart.line_items.includes(+ variant: [{ product: %i[store product_publications] }, { stock_levels: %i[stock_location active_stock_reservations] }]+ ).to_aenddef delivery_step_required?@@ -125,7 +129,13 @@ module Spree# What checkout has to collect, which is the whole total unless an# arrangement collects only part of it up front.def payment_satisfied?- @cart.payments.valid.sum(:amount) >= @cart.amount_due_at_checkout+ paid = if @cart.read_association_in_memory?(:payments)+ @cart.payments.reject(&:has_invalid_status?).sum(BigDecimal('0'), &:amount)+ else+ @cart.payments.valid.sum(:amount)+ end++ paid >= @cart.amount_due_at_checkoutend# Advisory rather than completion-only: the buyer has to be told thediff --git a/spree/core/lib/spree/reporting/default_vocabulary.rb b/spree/core/lib/spree/reporting/default_vocabulary.rbindex d28e35ea48..89cc7b2942 100644--- a/spree/core/lib/spree/reporting/default_vocabulary.rb+++ b/spree/core/lib/spree/reporting/default_vocabulary.rb@@ -541,7 +541,7 @@ module Spree# and tax category per row and throw all of it away.hydrate: lambda { |store, ids, _params|currency = Spree::Current.currency || store.default_currency- products = store.products.with_deleted.includes(:primary_media).where(id: ids)+ products = store.products.with_deleted.includes(:primary_media, default_variant: :prices).where(id: ids)products.to_h do |product|amount = product.default_variant&.amount_in(currency)[product.id, {diff --git a/spree/core/lib/spree/testing_support/factories/product_factory.rb b/spree/core/lib/spree/testing_support/factories/product_factory.rbindex 42ea50d9a1..ca1016b0f8 100644--- a/spree/core/lib/spree/testing_support/factories/product_factory.rb+++ b/spree/core/lib/spree/testing_support/factories/product_factory.rb@@ -15,8 +15,11 @@ FactoryBot.define doendafter(:create) do |product|if product.store&.default_channel && product.product_publications.empty?- Spree::ProductPublication.create!(- product: product,+ # Built through the association, not Spree::ProductPublication.create!:+ # the `empty?` above loads and caches the collection, so a row written+ # around it leaves the product holding an association that reads empty+ # while the table has the row.+ product.product_publications.create!(channel: product.store.default_channel,published_at: product.available_on,unpublished_at: product.discontinue_on@@ -69,8 +72,11 @@ FactoryBot.define do# keep passing. Production callers must publish explicitly via the# Admin SDK / Dashboard create form.if product.store&.default_channel && product.product_publications.empty?- Spree::ProductPublication.create!(- product: product,+ # Built through the association, not Spree::ProductPublication.create!:+ # the `empty?` above loads and caches the collection, so a row written+ # around it leaves the product holding an association that reads empty+ # while the table has the row.+ product.product_publications.create!(channel: product.store.default_channel,published_at: product.available_on,unpublished_at: product.discontinue_ondiff --git a/spree/core/spec/models/spree/base_spec.rb b/spree/core/spec/models/spree/base_spec.rbindex 81ff888429..1aac3f5b44 100644--- a/spree/core/spec/models/spree/base_spec.rb+++ b/spree/core/spec/models/spree/base_spec.rb@@ -180,4 +180,30 @@ describe Spree::Base doendend+ describe '.reset_counters_in_bulk' do+ let(:store) { @default_store }+ let(:collections) { create_list(:collection, 2, store: store) }+ let(:products) { create_list(:product, 3, store: store) }++ before do+ Spree::ProductCollection.insert_all([+ { collection_id: collections.first.id, product_id: products.first.id, position: 1 },+ { collection_id: collections.first.id, product_id: products.second.id, position: 2 },+ { collection_id: collections.second.id, product_id: products.first.id, position: 1 }+ ])+ Spree::Product.where(id: products.map(&:id)).update_all(collections_count: 9)+ end++ it 'recounts each record from its own rows, including the ones with none' do+ Spree::Product.reset_counters_in_bulk(products.map(&:id), :product_collections)++ expect(products.map { |product| product.reload.collections_count }).to eq([2, 1, 0])+ end++ it 'reads the counter column from the other side of the association' do+ Spree::Collection.reset_counters_in_bulk(collections.map(&:id), :product_collections)++ expect(collections.map { |collection| collection.reload.products_count }).to eq([2, 1])+ end+ endenddiff --git a/spree/core/spec/models/spree/stock/quantifier_spec.rb b/spree/core/spec/models/spree/stock/quantifier_spec.rbindex 6d6a59208b..c13b78f2d2 100644--- a/spree/core/spec/models/spree/stock/quantifier_spec.rb+++ b/spree/core/spec/models/spree/stock/quantifier_spec.rb@@ -58,6 +58,20 @@ module Spreeexpect(quantifier.available_stock).to eq(described_class.new(variant).available_stock)end+ it 'answers from memory for variants loaded through .preload' do+ loaded = Spree::Variant.find(variant.id)+ described_class.preload([loaded])+ quantifier = described_class.new(loaded)++ queries = count_queries do+ quantifier.available_stock+ quantifier.reserved_quantity+ end++ expect(queries).to eq(0)+ expect(quantifier.available_stock).to eq(described_class.new(variant).available_stock)+ end+it 'queries when the rows are not loaded' doquantifier = described_class.new(Spree::Variant.find(variant.id))diff --git a/spree/core/spec/models/spree/stock_location_spec.rb b/spree/core/spec/models/spree/stock_location_spec.rbindex 8b37a70e3e..db7e2e7128 100644--- a/spree/core/spec/models/spree/stock_location_spec.rb+++ b/spree/core/spec/models/spree/stock_location_spec.rb@@ -195,7 +195,7 @@ module Spreeit 'allocates a variant to a fulfillment' dofulfillment = double- expect(subject).to receive(:move).with(variant, 2, kind: 'allocated', cause: fulfillment)+ expect(subject).to receive(:move).with(variant, 2, kind: 'allocated', cause: fulfillment, stock_level: nil)subject.allocate(variant, 2, fulfillment)end@@ -206,7 +206,7 @@ module Spreeendit 'adjusts a variant with a reason' do- expect(subject).to receive(:move).with(variant, -2, kind: 'adjusted', reason: 'Damaged')+ expect(subject).to receive(:move).with(variant, -2, kind: 'adjusted', reason: 'Damaged', stock_level: nil)subject.adjust(variant, -2, reason: 'Damaged')enddiff --git a/spree/core/spree_core.gemspec b/spree/core/spree_core.gemspecindex 0fcba06a41..471ab72f1c 100644--- a/spree/core/spree_core.gemspec+++ b/spree/core/spree_core.gemspec@@ -84,7 +84,7 @@ Gem::Specification.new do |s|s.add_dependency 'wannabe_bool's.add_dependency 'safely_block', '>= 0.4', '< 2.0's.add_dependency 'phonelib', '~> 0.10'- s.add_dependency 'ar_lazy_preload', '~> 2.0'+ s.add_dependency 'ar_lazy_preload', '~> 2.1's.add_dependency 'bcrypt', '~> 3.1's.add_dependency 'sqids', '~> 0.2's.add_dependency 'ssrf_filter', '~> 1.0'