Skip to content

armgit / spree

Public Imported from GitHub
0

Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.

Queue and cap local test runs, leave the full suite to CI

Damian Legawiece5aaf9994905
diff --git a/.claude/commands/implement-plan.md b/.claude/commands/implement-plan.md
index 4d785000e2..cb53d6f597 100644
--- a/.claude/commands/implement-plan.md
+++ b/.claude/commands/implement-plan.md
@@ -1,5 +1,5 @@
---
-description: Deliver a docs/plans plan end to end — settle open questions, implement with local CI, two code reviews plus a simplify pass, running QA environment with seeded data, schema review, and an open, monitored pull request on spree/spree.
+description: Deliver a docs/plans plan end to end — settle open questions, implement with the changed specs green locally and the full suite on CI, two code reviews plus a simplify pass, running QA environment with seeded data, schema review, and an open, monitored pull request on spree/spree.
argument-hint: <plan filename or slug>
---
@@ -69,21 +69,25 @@ reason.
## 3. Implement
- Start with `/goal` so the run has a stated goal to check itself against:
- the deliverables from stage 1 plus green local CI.
+ the deliverables from stage 1 plus green changed specs and green CI.
- Follow the plan's phase order and every convention in `CLAUDE.md` (models,
migrations, API controllers, serializers, dashboard, translations in every
locale, type generation pipeline when serializers change, OpenAPI regenerated
from the integration specs — never edited by hand).
-- Test in two passes, always locally, always before moving on:
- 1. **Affected code first.** Run the specs for the files you changed (`bundle
- exec rspec <paths>` in the engine, `pnpm test` / `pnpm exec tsc -b` in the
- package). Iterate here until green — this is the fast loop.
- 2. **Then the full suite, the way CI runs it.** For every engine you touched
- (`spree/core`, `spree/api`, `spree/emails`, `spree/providers/*`,
- `spree/dashboard`, `spree/opentelemetry`): `bundle exec rake test_app`
- if the dummy app is stale, `bundle exec rake parallel_setup` after any
- schema change, then `bundle exec parallel_rspec spec`. Re-run any failing
- example on its own before investigating — confirm it really fails. For the
+- Test in two passes, always before moving on:
+ 1. **Affected code first, locally.** `pnpm test:changed` runs the specs for
+ the files this branch changed, engine by engine, at low priority
+ (`pnpm test:rspec <engine> <paths>` for a hand-picked set; `pnpm test` /
+ `pnpm exec tsc -b` in a package). Iterate here until green — this is the
+ fast loop. Read the "changed without a matching spec" list it prints: a
+ factory, a migration or a base class touches everything and is what the
+ next pass is for.
+ 2. **Then the full suite, on CI.** Push the branch and watch the checks
+ (`gh pr checks --watch` once the PR exists, or `gh run watch`). The laptop
+ runs many sessions at once; a full local suite is only for reproducing a
+ CI failure, and then it goes through `pnpm test:rspec <engine>`, which
+ queues behind the other sessions' suites. Re-run any failing example on
+ its own before investigating — confirm it really fails. For the
TypeScript packages: `pnpm turbo lint typecheck test build --force` from
the repo root (the `--force` matters — a cached run reads the SDK's stale
`dist`). If the dashboard changed, run the affected Playwright specs with
@@ -92,8 +96,8 @@ reason.
how; use `git commit --fixup` for follow-ups to a change and squash them
before opening the PR. No `Co-Authored-By` or "generated with" trailers.
-Done when: every deliverable from stage 1 exists, and both test passes are
-green.
+Done when: every deliverable from stage 1 exists, the changed specs are green
+locally and CI is green on the pushed branch.
## 4. Code review — first round
@@ -104,8 +108,8 @@ your notes for the final report, not silence.
## 5. Simplify
Run `/simplify` on the changed code. Apply the cleanups it proposes that keep
-behaviour identical; re-run the affected specs after, then the full suite for
-any engine or package the simplification touched.
+behaviour identical; re-run `pnpm test:changed` after, push, and let CI cover
+the rest of every engine or package the simplification touched.
## 6. Code review — second round
diff --git a/.claude/hooks/pre-commit-gate.sh b/.claude/hooks/pre-commit-gate.sh
index db61936049..ba92c377f5 100755
--- a/.claude/hooks/pre-commit-gate.sh
+++ b/.claude/hooks/pre-commit-gate.sh
@@ -17,6 +17,28 @@ set -u
# Parse the command out of the Claude hook input JSON.
cmd=$(jq -r '.tool_input.command // empty')
+# Full test suites are queued machine-wide and are otherwise CI's job (see
+# CLAUDE.md "Testing"). Block the bare forms so a session that has not read the
+# rule is redirected to the wrappers; targeted `rspec <paths>` runs pass.
+while IFS= read -r segment; do
+ seg="${segment#"${segment%%[![:space:]]*}"}"
+ case " $seg " in *" pnpm test:rspec "*|*"scripts/test/rspec "*|*" pnpm wt:e2e"*|*"scripts/worktree/e2e.sh"*) continue ;; esac
+ prefix='^([A-Za-z_][A-Za-z0-9_]*=[^[:space:]]*[[:space:]]+)*(env[[:space:]]+)?'
+ if [[ "$seg" =~ ${prefix}(bundle[[:space:]]+exec[[:space:]]+|bin/)?parallel_rspec([[:space:]]|$) ]] \
+ || [[ "$seg" =~ ${prefix}(bundle[[:space:]]+exec[[:space:]]+|bin/)?rspec([[:space:]]+spec/?)?[[:space:]]*$ ]] \
+ || [[ "$seg" =~ ${prefix}pnpm[[:space:]]+(-[^[:space:]]+[[:space:]]+)*test:e2e([[:space:]]|$) ]]; then
+ {
+ echo "Blocked: $seg"
+ echo "Full suites run one at a time on this machine and are CI's job. Use instead:"
+ echo " pnpm test:changed specs covering this branch's changed files"
+ echo " pnpm test:rspec <engine> [paths] one engine; a full suite is queued, capped and low priority"
+ echo " pnpm wt:e2e [spec] Playwright, queued the same way"
+ echo "Then push and watch CI for the full suite. If these commands are missing, run 'git merge main' first. See CLAUDE.md, Testing."
+ } >&2
+ exit 2
+ fi
+done < <(echo "$cmd" | tr ';&|' '\n')
+
# Split on shell separators (; && || | &) and check each segment for a real
# git commit/push invocation. Handles env-var prefixes (FOO=bar, env FOO=bar)
# and git's top-level option flags (-C path, -c k=v, --no-pager, etc.) so
diff --git a/CLAUDE.md b/CLAUDE.md
index c8aa4fb5e8..0173cf6aef 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -635,7 +635,7 @@ cd packages/sdk && pnpm test # 5. SDK tests
- Zod schemas → `packages/sdk/src/zod/generated/`
- Store types: `StoreProduct`, `StoreOrder`, etc. Admin types: `AdminProduct`, `AdminOrder`, etc.
-A **Lefthook pre-commit hook** (`lefthook.yml`) regenerates types and Zod schemas automatically whenever `spree/api/app/serializers/**/*.rb` files are committed, then re-stages the generated output. You don't need to run steps 1 and 2 manually if you're committing serializer changes — the hook handles it. Steps 3–5 (integration tests, OpenAPI regen, SDK tests) still need to run locally before pushing.
+A **Lefthook pre-commit hook** (`lefthook.yml`) regenerates types and Zod schemas automatically whenever `spree/api/app/serializers/**/*.rb` files are committed, then re-stages the generated output. You don't need to run steps 1 and 2 manually if you're committing serializer changes — the hook handles it. Steps 3–5 (integration tests, OpenAPI regen, SDK tests) still need to run locally before pushing — run step 3 as `pnpm test:rspec api spec/integration/` so it queues with the other sessions.
### Changesets & Versioning
@@ -645,17 +645,28 @@ Published packages use **Changesets** for versioning — one workspace-wide inst
## Testing
-Always run tests before committing changes.
+Run the specs for what you changed before committing, then push and let CI run the full suite. This machine runs many sessions at once, so **never start a bare `parallel_rspec`** — every test run goes through `scripts/test/`:
+
+```bash
+pnpm test:changed # specs covering this branch's changed files (vs main + uncommitted), engine by engine
+pnpm test:changed -- --list # only show what would run, and which changed files have no spec
+pnpm test:rspec core spec/models/spree/state_spec.rb:7 # a targeted run in one engine (low priority, no queue)
+pnpm test:rspec api # a full engine suite: queued behind other suites, one worker per performance core, low priority
+pnpm test:slots # who holds the machine-wide test slot right now
+```
+
+`scripts/test/rspec <engine>` rebuilds the dummy app when it is missing or older than a migration, refreshes the worker databases, and splits a targeted run of several files across up to four workers, so `test_app` / `parallel_setup` rarely need running by hand. Engines: `core`, `api`, `emails`, `dashboard`, `opentelemetry`, `easypost`, `meilisearch`, `stripe`.
+
+**The full suite is CI's job, not the laptop's.** The repository is public, GitHub-hosted runners are free, and `tests.yml` already shards core twelve ways. Push and watch the checks (`gh pr checks --watch`, or `/autofix-pr`). Run a full suite locally only when CI reports a failure you need to reproduce, or when `test:changed` lists a change with no matching spec that touches everything (a factory, a migration, `Spree::Base`). One suite runs at a time machine-wide (`SPREE_TEST_SLOTS` to allow more); queued runs print who they are waiting for. Runs use the `utility` scheduling class, which yields to interactive work but may use every core; `SPREE_TEST_QOS=background` confines a run to the efficiency cores when the machine must stay cool, and `SPREE_TEST_WORKERS` changes the worker count. **A queue of sessions waiting on the slot means they are running full suites they should have pushed to CI.**
### Backend (Ruby — RSpec)
-Each engine has its own test suite:
+Each engine has its own test suite. Under the wrappers above, the raw commands are:
```bash
cd spree && bundle install # shared deps
cd core && bundle install # engine deps
bundle exec rake test_app # create dummy Rails app (skip if already exists)
-bundle exec rspec # run full suite
bundle exec rspec spec/models/spree/state_spec.rb # single file
bundle exec rspec spec/models/spree/state_spec.rb:7 # single test
```
@@ -666,15 +677,7 @@ Default DB is SQLite3. For PostgreSQL:
DB=postgres DB_USERNAME=postgres DB_PASSWORD=password DB_HOST=localhost bundle exec rake test_app
```
-**Parallel runs:**
-
-```bash
-bundle exec rake parallel_setup # create worker DBs
-bundle exec parallel_rspec spec # run in parallel
-bundle exec parallel_rspec -n 4 spec # with worker count
-```
-
-Re-run `parallel_setup` after schema changes.
+Re-run `parallel_setup` after schema changes (`scripts/test/rspec <engine>` does this before every full suite).
**Test guidelines:**
@@ -682,7 +685,7 @@ Re-run `parallel_setup` after schema changes.
- Prefer `build` over `create` for speed, use `create_list` for creating multiple records
- Factories live in `lib/spree/testing_support/factories/`
- ALWAYS use factories in tests, never call `Model#create` directly
-- ALWAYS run parallel tests if running full test suite, if there are any failures repeat the failed examples seperately and confirm they really fail before investigating
+- A full suite runs through `scripts/test/rspec <engine>` (queued, capped, parallel); if there are any failures repeat the failed examples separately and confirm they really fail before investigating
- Pragmatic — no tests for standard Rails validations, only custom ones
- Controller specs: always add `render_views`, use `stub_authorization!` for auth
- Use controller specs for testing edge cases, API integration tests are only for happy path/simple 422 failures to generate OpenAPI examples; otherwise they get too brittle and high-maintenance
diff --git a/docs/plans/6.1-vies-vat-validation.md b/docs/plans/6.1-vies-vat-validation.md
new file mode 100644
index 0000000000..c1cbe30e30
--- /dev/null
+++ b/docs/plans/6.1-vies-vat-validation.md
@@ -0,0 +1,248 @@
+# VIES VAT Registry Validation
+
+**Status:** Design Finalized (open questions resolved 2026-09-05)
+**Target:** Spree 6.1
+**Depends on:** `6.0-tax-provider.md` (shipped `Spree::TaxIdentifier` + `Spree.tax_id_validators` registry, `TaxIdentifiers::Validate` service, `ValidateJob`, validation subscriber, `POST …/tax_identifiers/:id/validate`); `6.0-b2b-companies-and-catalogs.md` (company-owned identifiers, `legal_entity` resolution)
+**Author:** Damian + Claude
+**Last updated:** 2026-09-05
+
+## Summary
+
+Ship the **registry half** of EU VAT number validation as a free open-source
+gem: a validator that asks VIES (the EU's VAT Information Exchange System)
+whether a number is actually registered, records audit-grade evidence — the
+consultation number a qualified check returns — and keeps verdicts fresh over
+time. Core 6.0 deliberately shipped only the offline half
+(`Spree::TaxIdentifiers::Validator::EuVat`: format + checksum via valvat,
+`checks_registry?` false) with a documented contract that an extension
+re-registers `eu_vat` and takes over both halves; this plan is that
+extension. The pipeline — statuses (`pending / verified / unverified /
+unavailable / unsupported`), async job, subscriber, admin validate endpoint —
+already exists and starts doing real work the moment the gem registers.
+
+Free because compliance plumbing cannot be a paid feature (the same reasoning
+that put exemption certificates in OSS): the underlying API is free and
+credential-less, the strongest precedent ships it free in open source, and
+closed SaaS platforms validate at checkout as a built-in — "real VIES
+validation with audit-grade evidence, free" extends the 6.0 B2B headline.
+The Enterprise tier keeps its distance: governance and commercial terms,
+never compliance.
+
+## Key Decisions (do not deviate without discussion)
+
+- **A gem, never core.** Reaffirming `6.0-tax-provider.md`: no registry
+ client in core. The gem re-registers the `eu_vat` kind with a validator
+ whose `checks_registry?` is true and which keeps the offline half as its
+ first step (format-invalid fails fast as `unverified`, no network call).
+- **valvat's lookup client is the transport.** Core already depends on
+ valvat for checksums and deliberately avoids loading its lookup half; the
+ gem uses it. valvat supports the qualified request (`requester:` option)
+ and surfaces the consultation number and trader details — hand-rolling a
+ VIES client would duplicate a battle-tested one we already ship.
+- **Qualified checks whenever possible — the consultation number is the
+ point.** A plain check answers valid/invalid; a qualified check (supplying
+ the merchant's own VAT number) returns a **consultation number**, the
+ official proof that this number was verified on that date — what defends
+ zero-rating a reverse-charge sale in an audit. When the store has no
+ requester VAT number configured, the gem downgrades to plain checks and
+ says so in the evidence.
+- **The merchant's own registrations become first-class: `store` is the
+ fifth `TaxIdentifier` owner (2026-09-05).** A small core 6.1 change adds
+ `store_id` to `spree_tax_identifiers` (exactly-one-owner validation
+ extended, `(store_id, kind)` unique) with admin CRUD and a "Your tax
+ registrations" card in dashboard tax settings. The gem reads its requester
+ as `store.tax_identifiers` of kind `eu_vat` — no preference field —
+ and the merchant's own number flows through the same validation pipeline
+ as everyone else's. This is also the durable home future merchant-side
+ registrations (OSS/IOSS, GB) were always going to need.
+- **Evidence is persisted, not just a status.** Every registry answer writes
+ the identifier's `validation_status`/`validated_at` (as today) **and** an
+ evidence record in the identifier's `metadata` under a `vies` key: verdict,
+ checked-at, consultation number, requester VAT used, and the trader
+ name/address VIES returned — plus a capped history (last 10 checks),
+ because the audit question is "was it valid *when you zero-rated it*",
+ which the current-status columns alone cannot answer.
+- **`invalid` and `couldn't ask` never blur.** VIES proxies 27 national
+ registries and individual member states go down routinely
+ (`MS_UNAVAILABLE`), rate limits included. A registry that cannot answer
+ yields `unavailable` (the base contract already demands this) and the
+ check is re-queued with backoff; only an actual "not registered" answer
+ yields `unverified`. The gem never turns an outage into a rejection.
+- **Verdicts are data; tax policy stays with tax providers.** The gem
+ delivers statuses and evidence. Whether an unverified-but-format-valid
+ number still earns reverse charge during a VIES outage is a decision for
+ the consuming tax provider — this gem takes no checkout-blocking stance
+ (deliberately unlike the SaaS precedent, which blocks with no override).
+- **Name/address matching is a soft signal, never a gate.** Returned trader
+ details are stored and surfaced (a "registered name differs" hint in the
+ dashboard) but never fail a validation — several member states (Germany
+ foremost) return no trader details at all.
+- **EU first; GB is a fast-follow kind in the same gem (2026-09-05).**
+ `SpreeVies::HmrcValidator` registers for `gb_vat` right after the VIES
+ core ships — HMRC's free check-VAT API follows the same evidence pattern
+ (it returns a consultation number too). Switzerland and Norway arrive
+ later as further kinds — nothing in this gem's design may assume VIES is
+ the only registry.
+
+## Design Details
+
+### The validator
+
+```ruby
+module SpreeVies
+ class Validator < Spree::TaxIdentifiers::Validator::EuVat
+ def self.checks_registry? = true
+
+ # Called by Spree::TaxIdentifiers::Validate (unchanged core service).
+ # 1. Offline half first — inherited valid_format?; a malformed number is
+ # 'unverified' with no network call.
+ # 2. Qualified VIES lookup via Valvat::Lookup (requester: store VAT when
+ # configured), mapped to a ValidationResult:
+ # registered -> 'verified' (+ evidence)
+ # not registered -> 'unverified' (+ evidence)
+ # MS_UNAVAILABLE, timeout, rate-limit -> 'unavailable' (retryable)
+ def self.check_registry(tax_identifier)
+ # ...
+ end
+ end
+end
+
+# engine initializer
+Spree.tax_id_validators['eu_vat'] = 'SpreeVies::Validator'
+```
+
+The core `Validate` service already maps raised transport errors to an
+`unavailable` result; the validator only raises typed errors for genuinely
+unexpected failures.
+
+### Evidence shape (`tax_identifier.metadata['vies']`)
+
+```json
+{
+ "last": {
+ "status": "verified",
+ "checked_at": "2026-09-05T10:12:00Z",
+ "consultation_number": "WAPIAAAAY8kFxxxx",
+ "requester_vat": "PL5252242171",
+ "name": "ACME GMBH",
+ "address": "HAUPTSTR. 1, 10115 BERLIN"
+ },
+ "history": [ { "...": "last 10 entries, newest first" } ]
+}
+```
+
+Not columns: the shape is registry-specific, and the HMRC/CH/NO validators
+will write their own keys. The columns stay what they are today — the
+cross-registry summary (`validation_status`, `validated_at`).
+
+### Requester VAT number — store-owned TaxIdentifier (core 6.1 change)
+
+Core gains `store` as the fifth identifier owner: `store_id` on
+`spree_tax_identifiers`, the exactly-one-owner validation extended,
+`(store_id, kind)` unique index, `Spree::Store has_many :tax_identifiers`,
+admin CRUD (`/api/v3/admin/tax_identifiers`, store-owned rows only, under
+the `settings` scope) and a "Your tax registrations" card in dashboard tax
+settings. The merchant's own numbers run through the same validation
+pipeline as buyers' numbers — a merchant with an invalid registration finds
+out from their own dashboard.
+
+The gem resolves its requester as the store's `eu_vat` identifier. Present →
+every check is qualified and earns a consultation number; absent → plain
+checks, and the evidence records `"qualified": false` so the admin UI can
+nudge ("add your VAT registration to get audit evidence").
+
+### Resilience & freshness
+
+- **Retries:** `unavailable` results re-queue the existing `ValidateJob`
+ with exponential backoff (minutes → hours, capped attempts per day);
+ VIES's own rate-limit responses are honored with a global cool-off.
+- **Throttling:** one identifier is never checked twice concurrently
+ (the job already runs per-identifier; the gem adds an in-flight guard).
+- **Re-validation:** numbers get deregistered, so `verified` decays — a
+ recurring task (`spree_vies:revalidate`, wired as a Solid Queue recurring
+ job in the starter, plain rake elsewhere) re-enqueues identifiers whose
+ `validated_at` is older than the freshness window (default 90 days,
+ gem-configurable). On-demand re-checks stay on the existing admin
+ endpoint.
+
+### Dashboard
+
+Small additions to the existing tax registration card (company page and
+customer page): the status badge gains evidence detail — "Verified via VIES
+on ⟨date⟩ · consultation #⟨n⟩", registered name/address when returned, a
+soft "differs from company name" hint, a re-validate button (existing
+endpoint), and the requester-VAT nudge when checks are unqualified. Plus the
+requester VAT field in tax settings. All behind data the serializer already
+carries (`metadata`).
+
+### Testing
+
+Client stubbed with WebMock fixtures for the full verdict matrix (valid,
+invalid, MS_UNAVAILABLE, timeout, rate-limited, qualified vs plain) — never
+live VIES in CI. valvat's own test helpers cover lookup stubbing.
+
+## Migration Path
+
+One additive core migration (`store_id` on `spree_tax_identifiers`); no data
+tasks, no upgrade-manifest step. Phases:
+
+0. **Core: store-owned identifiers** — fifth owner (`store_id`, exactly-one
+ validation, `(store_id, kind)` unique), `Store has_many`, admin CRUD +
+ serializer, dashboard "Your tax registrations" card in tax settings.
+1. **Gem skeleton** — `spree/providers/vies` (`spree_vies`) engine with the
+ validator, registry re-registration, verdict mapping, evidence
+ persistence.
+2. **Resilience** — backoff/re-queue on `unavailable`, rate-limit cool-off,
+ in-flight guard.
+3. **Qualified checks** — requester resolution from the store's `eu_vat`
+ identifier, evidence `qualified` flag.
+4. **Freshness** — `spree_vies:revalidate` recurring task (default 90 days,
+ `SpreeVies.freshness` configurable), wired as a Solid Queue recurring job
+ in the starter.
+5. **Dashboard** — evidence on the registration card, re-validate
+ affordance, requester nudge.
+6. **Docs** — how verification interacts with reverse charge and exemptions;
+ the "verdicts are data" guidance for tax-provider authors.
+7. **GB fast-follow** — `SpreeVies::HmrcValidator` for the `gb_vat` kind
+ (HMRC check-VAT REST, its consultation number in evidence under an
+ `hmrc` key).
+
+## Constraints on Current Work
+
+- **Core's `Validator::EuVat` stays the offline half and keeps working
+ standalone** — no VIES calls, no valvat lookup loading in core, ever.
+- **Nothing may treat `validation_status` as a checkout gate.** Consumers
+ (tax providers, exemption resolution) read it as evidence quality:
+ `unavailable`/`pending` mean *not yet known*, never *invalid*. Blocking
+ behavior, if a merchant wants it, belongs to the tax provider's
+ configuration — not to core, not to this gem.
+- **Registry evidence lives under a registry-specific `metadata` key**, and
+ new registry validators follow the same pattern — no evidence columns on
+ `spree_tax_identifiers`.
+- **A new registry (GB, CH, NO) is a new validator kind in
+ `Spree.tax_id_validators`** — never a special case inside the VIES
+ validator.
+
+## Open Questions
+
+All four design questions (gem location, requester VAT source, freshness
+window, GB scope) were resolved 2026-09-05 and folded into Key Decisions:
+monorepo `spree/providers/vies`; store-owned `TaxIdentifier` as the fifth
+owner; 90-day freshness default; GB as a fast-follow kind in the same gem.
+Nothing is currently open.
+
+## References
+
+- Core contract this fills: `spree/core/app/models/spree/tax_identifiers/validator/eu_vat.rb`
+ (`checks_registry?` and its documentation), `validator/base.rb`,
+ `spree/core/app/services/spree/tax_identifiers/validate.rb`,
+ `spree/core/app/jobs/spree/tax_identifiers/validate_job.rb`
+- `6.0-tax-provider.md` — the validator-registry decision ("NOT a provider
+ method, no registry client in core"); `6.0-b2b-companies-and-catalogs.md`
+ — identifiers on legal-entity nodes, `legal_entity` resolution
+- VIES REST/SOAP services (credential-less); valvat lookup (`requester:` →
+ consultation number); HMRC check-VAT API (GB follow-on)
+- Precedent: the leading open-source suite ships built-in VIES validation
+ free with automatic group assignment; the leading SaaS validates at
+ checkout with no override on outage — the resilience stance above is the
+ deliberate contrast.
diff --git a/package.json b/package.json
index c966bb7276..6b14ec0f7a 100644
--- a/package.json
+++ b/package.json
@@ -34,6 +34,9 @@
"wt:seller": "scripts/worktree/dev-seller.sh",
"wt:storefront": "scripts/worktree/dev-storefront.sh",
"wt:e2e": "scripts/worktree/e2e.sh",
+ "test:changed": "scripts/test/changed",
+ "test:rspec": "scripts/test/rspec",
+ "test:slots": "scripts/test/with-slot --status",
"docs:dev": "cd docs && mint dev --port 3333",
"changeset": "changeset",
"version:preview": "changeset version --ignore @spree/sdk"
diff --git a/scripts/test/changed b/scripts/test/changed
new file mode 100755
index 0000000000..0d293a03db
--- /dev/null
+++ b/scripts/test/changed
@@ -0,0 +1,123 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Runs the specs that cover the files changed on this branch, engine by engine,
+# through scripts/test/rspec. This is the local pass before a push; the full
+# suite runs on CI, which is free for this public repository and already
+# sharded twelve ways.
+#
+# scripts/test/changed # changes since the merge base with main, plus uncommitted work
+# scripts/test/changed --list # only print what would run
+# SPREE_TEST_BASE=origin/main scripts/test/changed
+#
+# Files the mapping cannot place (factories, migrations, support files) are
+# listed so you can decide whether they need the full suite.
+
+require 'set'
+
+ROOT = `git rev-parse --show-toplevel`.strip
+BASE = ENV.fetch('SPREE_TEST_BASE', 'main')
+LIST_ONLY = ARGV.include?('--list')
+
+def changed_files
+ merge_base = `git merge-base #{BASE} HEAD`.strip
+ committed = `git diff --name-only #{merge_base}...HEAD`.lines
+ working = `git diff --name-only HEAD`.lines
+ untracked = `git ls-files --others --exclude-standard`.lines
+ (committed + working + untracked).map(&:strip).reject(&:empty?).uniq
+end
+
+# Splits a monorepo path into the engine directory and the path inside it.
+def split_engine(path)
+ match = path.match(%r{\A(spree/(?:providers/)?[^/]+)/(.+)\z})
+ return nil unless match && File.directory?(File.join(ROOT, match[1], 'spec'))
+
+ [match[1], match[2]]
+end
+
+def engine_glob(engine, pattern)
+ Dir.glob(File.join(ROOT, engine, pattern)).map { |file| file.delete_prefix("#{ROOT}/#{engine}/") }
+end
+
+def specs_for(engine, relative)
+ candidates =
+ case relative
+ when %r{\Aspec/.+_spec\.rb\z} then [relative]
+ when %r{\Aspec/} then []
+ when %r{\Aapp/(.+)\.rb\z} then ["spec/#{Regexp.last_match(1)}_spec.rb"]
+ when %r{\Alib/(.+)\.rb\z} then ["spec/lib/#{Regexp.last_match(1)}_spec.rb"]
+ when %r{\Aconfig/locales/} then ['spec/i18n_spec.rb']
+ else []
+ end
+
+ # An API controller is also exercised by the integration spec of the same
+ # name (and the nested ones under it); a serializer by the integration specs
+ # of its namespace that mention its resource.
+ if (controller = relative[%r{\Aapp/controllers/(spree/api/v3/.+)_controller\.rb\z}, 1])
+ candidates << "spec/integration/#{controller}_spec.rb"
+ candidates += engine_glob(engine, "spec/integration/#{controller}/**/*_spec.rb")
+ elsif (match = relative.match(%r{\Aapp/serializers/spree/api/v3/(?:(admin|seller)/)?(?:.+/)?([a-z_]+)_serializer\.rb\z}))
+ namespace = match[1] || 'store'
+ candidates += engine_glob(engine, "spec/integration/spree/api/v3/#{namespace}/**/*#{match[2]}*_spec.rb")
+ end
+
+ candidates.uniq.select { |spec| File.exist?(File.join(ROOT, engine, spec)) }
+end
+
+specs = Hash.new { |hash, key| hash[key] = Set.new }
+unmapped = []
+other = []
+
+changed_files.each do |path|
+ next if path.end_with?('.md') || path.end_with?('.mdx')
+
+ engine, relative = split_engine(path)
+ if engine.nil?
+ other << path
+ next
+ end
+
+ found = specs_for(engine, relative)
+ if found.empty?
+ unmapped << path unless relative.start_with?('spec/dummy/', 'spec/support/')
+ else
+ specs[engine].merge(found)
+ end
+end
+
+if specs.empty?
+ puts 'No changed Ruby files map to specs.'
+else
+ specs.each do |engine, files|
+ puts "▸ #{engine}: #{files.size} spec file(s)"
+ files.sort.each { |file| puts " #{file}" }
+ end
+end
+
+unless unmapped.empty?
+ puts "▸ changed without a matching spec (CI runs the full suite for these):"
+ unmapped.sort.each { |file| puts " #{file}" }
+end
+
+if other.any? { |path| path.start_with?('packages/') }
+ puts '▸ packages/ changed: run `pnpm turbo lint typecheck test build --force` from the repo root'
+end
+if specs.key?('spree/core') && !specs.key?('spree/api')
+ puts '▸ core changed: the api suite is affected too and runs on CI'
+end
+
+exit if LIST_ONLY || specs.empty?
+
+failed = []
+specs.each do |engine, files|
+ name = File.basename(engine)
+ puts "\n▸ running #{name}"
+ system(File.join(ROOT, 'scripts', 'test', 'rspec'), name, *files.sort) || failed << name
+end
+
+if failed.empty?
+ puts "\n✓ changed specs green"
+else
+ puts "\n✗ failures in: #{failed.join(', ')}"
+ exit 1
+end
diff --git a/scripts/test/rspec b/scripts/test/rspec
new file mode 100755
index 0000000000..90bced7982
--- /dev/null
+++ b/scripts/test/rspec
@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+# Run an engine's specs without fighting the other sessions on this machine.
+#
+# scripts/test/rspec core # full suite: queued behind other suites,
+# # capped workers, background priority
+# scripts/test/rspec api spec/controllers/spree/api/v3/admin/orders_controller_spec.rb:12
+# # targeted: background priority, no queue
+#
+# Engines: core, api, emails, dashboard, opentelemetry, easypost, meilisearch, stripe.
+# Everything after the engine is passed to rspec; with nothing after it the whole
+# suite runs through parallel_rspec.
+#
+# SPREE_TEST_WORKERS parallel_rspec workers for a full suite (default: the number of
+# performance cores on Apple Silicon, otherwise all cores)
+# SPREE_TEST_TARGET_WORKERS
+# workers for a targeted run of several files (default 4)
+# SPREE_TEST_QOS macOS scheduling clamp: utility (yields to interactive work but may
+# use every core), background (efficiency cores only), maintenance,
+# or none (default utility)
+# SPREE_TEST_SLOTS how many full suites may run machine-wide at once (default 1)
+set -euo pipefail
+
+here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+root="$(git -C "$here" rev-parse --show-toplevel)"
+
+engine="${1:-}"
+[ -n "$engine" ] || { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' >&2; exit 64; }
+shift
+
+if [ -d "$root/spree/$engine/spec" ]; then
+ engine_dir="$root/spree/$engine"
+elif [ -d "$root/spree/providers/$engine/spec" ]; then
+ engine_dir="$root/spree/providers/$engine"
+else
+ echo "scripts/test/rspec: unknown engine '$engine' (no spree/$engine/spec or spree/providers/$engine/spec)" >&2
+ exit 64
+fi
+
+default_workers() {
+ sysctl -n hw.perflevel0.logicalcpu 2>/dev/null \
+ || sysctl -n hw.ncpu 2>/dev/null \
+ || nproc 2>/dev/null \
+ || echo 4
+}
+workers="${SPREE_TEST_WORKERS:-$(default_workers)}"
+qos="${SPREE_TEST_QOS:-utility}"
+
+priority=()
+if [ "$qos" != "none" ]; then
+ if command -v taskpolicy >/dev/null; then
+ priority=(taskpolicy -c "$qos")
+ elif command -v nice >/dev/null; then
+ priority=(nice -n 19)
+ fi
+fi
+
+cd "$engine_dir"
+
+# The dummy app is rebuilt when missing, and when any migration (the engine's own
+# or core's, which every engine loads) is newer than the schema it was built
+# from. A migration edited in place has its version recorded already, so
+# db:migrate would not repair that; only a rebuild does.
+newest_migration() {
+ find "$root/spree/core/db/migrate" db/migrate -name '*.rb' -newer spec/dummy/db/schema.rb 2>/dev/null | head -1
+}
+if [ ! -f spec/dummy/config/database.yml ]; then
+ echo "▸ spec/dummy missing in $engine_dir, building it (bundle exec rake test_app)"
+ bundle exec rake test_app >/dev/null
+elif [ ! -f spec/dummy/db/schema.rb ] || [ -n "$(newest_migration)" ]; then
+ echo "▸ spec/dummy in $engine_dir is older than a migration, rebuilding it (bundle exec rake test_app)"
+ bundle exec rake test_app >/dev/null
+fi
+
+# A targeted run. One or two files, a line filter or an rspec option run in a
+# single process; a larger set is split across workers, since a changed-files
+# list for a big branch can hold dozens of the heaviest specs.
+if [ $# -gt 0 ]; then
+ spec_files=0
+ plain_paths=1
+ for argument in "$@"; do
+ case "$argument" in
+ -*|*:[0-9]*) plain_paths=0 ;;
+ *) if [ -d "$argument" ]; then
+ spec_files=$((spec_files + $(find "$argument" -name '*_spec.rb' | wc -l)))
+ else
+ spec_files=$((spec_files + 1))
+ fi ;;
+ esac
+ done
+ target_workers="${SPREE_TEST_TARGET_WORKERS:-4}"
+ [ "$spec_files" -lt "$target_workers" ] && target_workers=$spec_files
+ if [ $plain_paths -eq 1 ] && [ "$target_workers" -ge 2 ]; then
+ bundle exec rake "parallel_setup[$target_workers]" >/dev/null
+ echo "▸ $engine: $spec_files spec files across $target_workers workers, qos=$qos"
+ exec "${priority[@]}" bundle exec parallel_rspec -n "$target_workers" "$@"
+ fi
+ exec "${priority[@]}" bundle exec rspec "$@"
+fi
+
+# A full suite: fresh worker databases (a file copy on SQLite), then queue behind
+# whatever other suite is running on this machine.
+bundle exec rake "parallel_setup[$workers]" >/dev/null
+echo "▸ full $engine suite: $workers workers, qos=$qos, waiting for a free test slot if needed"
+exec "$here/with-slot" -- "${priority[@]}" bundle exec parallel_rspec -n "$workers" spec
diff --git a/scripts/test/with-slot b/scripts/test/with-slot
new file mode 100755
index 0000000000..fe95f40df7
--- /dev/null
+++ b/scripts/test/with-slot
@@ -0,0 +1,99 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Runs a command inside one of a fixed number of machine-wide test slots.
+#
+# Several sessions each starting a full suite oversubscribe the CPU many times
+# over, and the SQLite test databases make concurrent suites fail with bogus
+# errors. Queueing them here keeps total throughput the same while the laptop
+# stays cool and the failures stay real.
+#
+# scripts/test/with-slot -- bundle exec parallel_rspec spec
+# scripts/test/with-slot --status
+#
+# SPREE_TEST_SLOTS how many suites may run at once (default 1)
+# SPREE_TEST_SLOT_DIR where the lock files live (default ~/.cache/spree-test-slots)
+
+require 'fileutils'
+require 'json'
+require 'time'
+
+SLOTS = Integer(ENV.fetch('SPREE_TEST_SLOTS', '1'))
+SLOT_DIR = ENV.fetch('SPREE_TEST_SLOT_DIR') { File.join(Dir.home, '.cache', 'spree-test-slots') }
+POLL_SECONDS = 2
+
+FileUtils.mkdir_p(SLOT_DIR)
+
+def slot_path(index)
+ File.join(SLOT_DIR, "slot-#{index}.lock")
+end
+
+def holder(index)
+ JSON.parse(File.read(slot_path(index)))
+rescue StandardError
+ nil
+end
+
+def describe_holder(index)
+ info = holder(index)
+ return 'held' unless info
+
+ "pid #{info['pid']} in #{info['cwd']} since #{info['started_at']}: #{info['command']}"
+end
+
+def status
+ (0...SLOTS).each do |index|
+ File.open(slot_path(index), File::RDWR | File::CREAT, 0o644) do |file|
+ if file.flock(File::LOCK_EX | File::LOCK_NB)
+ file.flock(File::LOCK_UN)
+ puts "slot #{index}: free"
+ else
+ puts "slot #{index}: #{describe_holder(index)}"
+ end
+ end
+ end
+end
+
+def acquire(command)
+ files = (0...SLOTS).map { |index| File.open(slot_path(index), File::RDWR | File::CREAT, 0o644) }
+ announced = false
+
+ loop do
+ file = files.find { |candidate| candidate.flock(File::LOCK_EX | File::LOCK_NB) }
+ if file
+ file.truncate(0)
+ file.write(JSON.generate(pid: Process.pid, cwd: Dir.pwd, started_at: Time.now.iso8601, command: command.join(' ')))
+ file.flush
+ warn "with-slot: slot acquired" if announced
+ return file
+ end
+
+ unless announced
+ warn "with-slot: all #{SLOTS} test slot(s) busy, waiting (scripts/test/with-slot --status to see who holds them)"
+ announced = true
+ end
+ sleep POLL_SECONDS
+ end
+end
+
+if ARGV == ['--status']
+ status
+ exit
+end
+
+command = ARGV.dup
+command.shift if command.first == '--'
+abort 'usage: scripts/test/with-slot [--] <command...>' if command.empty?
+
+slot = acquire(command)
+child = Process.spawn(*command)
+%w[INT TERM HUP].each do |signal|
+ trap(signal) do
+ Process.kill(signal, child)
+ rescue Errno::ESRCH
+ nil
+ end
+end
+Process.wait(child)
+slot.flock(File::LOCK_UN)
+exit($?.exitstatus || 1)
diff --git a/scripts/worktree/e2e.sh b/scripts/worktree/e2e.sh
index a3b285624f..11e0d3ba4c 100755
--- a/scripts/worktree/e2e.sh
+++ b/scripts/worktree/e2e.sh
@@ -1,7 +1,9 @@
#!/usr/bin/env bash
# Run the dashboard Playwright suite on this worktree's deterministic port
-# block, so multiple worktrees can run e2e concurrently. The suite already
-# isolates its database (per-worktree SQLite in spree/api/spec/dummy).
+# block. The suite isolates its database (per-worktree SQLite in
+# spree/api/spec/dummy) but boots Rails, Vite and browsers, so it takes the same
+# machine-wide slot as a full rspec suite (scripts/test/with-slot) instead of
+# competing with one.
# Extra args pass through to playwright: scripts/worktree/e2e.sh e2e/auth.spec.ts
set -euo pipefail
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib.sh"
@@ -18,4 +20,4 @@ export E2E_RAILS_PORT E2E_VITE_PORT
echo "▸ e2e on rails :$E2E_RAILS_PORT / vite :$E2E_VITE_PORT"
cd "$(worktree_root)/packages/dashboard"
-exec pnpm test:e2e "$@"
+exec "$(worktree_root)/scripts/test/with-slot" -- pnpm test:e2e "$@"