Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.
Preferences Stored as JSON, Secrets Encrypted
Status: In Progress — everything this plan targets for 6.0 is built (JSON coder, encrypted secret column, conversion migration, upgrade tasks, tiered calculators, docs); waiting on review and CI
Target: Spree 6.0
Depends on: nothing. Follows the precedent of the metadata consolidation (20260809120001_consolidate_metadata_columns.rb)
Author: Damian + Claude
Last updated: 2026-10-01 (implementation)
Tracking: Linear V-3730 · GitHub spree/spree#14786
Summary
Every model's preferences column is a YAML document, written by one line in Spree::Preferences::Preferable (serialize :preferences, type: Hash, coder: YAML). This plan moves them to a JSON column (jsonb on PostgreSQL, json on MySQL and SQLite) in 6.0, converting the stored data during the 6.0 upgrade. At the same time, preferences declared :password (gateway and integration secrets) move out of preferences into an encrypted secret_preferences column, so the main column never holds a secret. Declarations do not change: preference :secret_key, :password and preferred_secret_key work as before.
Reasons, in order: gateway secrets — including live Stripe secret keys — sit in the database in plain text today; loading YAML is an attack surface Rails has to fence with yaml_column_permitted_classes, a list Spree injects into every application and every Rails upgrade touches; and two tables built in 6.0 already declare preferences as JSON but, through the shared coder, hold a YAML document wrapped in a JSON string. Being able to query or index preferences is a side benefit only — nothing queries them today. 6.0 is a clean-break release and already carries a data-converting migration of the same shape, which makes it the right release for this.
Inventory
Measured on main at 2026-10-01. Smaller than expected: there is no log entry table any more, and global configuration (Spree::Config) is held in memory and environment variables, not in the database.
| Column | Today | Notes |
|---|---|---|
preferences on spree_calculators, spree_payment_methods, spree_promotion_rules, spree_stores, spree_integrations, spree_imports, spree_exports, spree_price_rules, spree_channels, spree_order_routing_rules, spree_seller_requirements |
text, YAML |
The main body of the work |
preferences on spree_delivery_method_rules, spree_commission_rules |
jsonb/json column holding a YAML string |
Already the right column type; only the data is wrong |
spree_preferences.value |
text, YAML |
One live row: the installation id. The table is dropped without copying it; the default store generates a new id in an internal install_id preference when it is next saved (settled during implementation) |
spree_gateways.preferences |
text |
Leftover 4.3 table, no model reads it. Not touched: the migration converts only tables a model storing Spree preferences reads, so a table another library owns is never changed |
Spree::Address serialize :preferences |
declaration only | spree_addresses has no such column. Kept, with the JSON coder: removing it changes the address attribute set that duplicate detection compares |
metadata is already JSON (Spree::Metadata::HashSerializer) and is out of scope. No provider gem (spree_stripe, spree_easypost, spree_meilisearch) has a preferences table of its own.
Values found in the YAML today, by declared preference type (counted across core and providers): string 69, boolean 58, integer 54, decimal 36 (stored as BigDecimal), password 7 (Stripe publishable key, secret key and two webhook signing secrets; EasyPost API key and webhook secret; the test gateway's key), array 16, hash 4, any 3, date 2 (already stored as a yyyy-MM-dd string), text 1. No datetime preference is declared anywhere. Keys are symbols.
Key Decisions (do not deviate without discussion)
- One JSON coder replaces YAML in
Preferable. It writes a plain JSON object with string keys and loads it as aHashWithIndifferentAccess, sopreferences[:amount],fetch(:amount)and the generatedpreferred_*readers keep working unchanged. - Decimals and times are restored on read from their declared type (narrowed 2026-10-01 during implementation). A
:decimalpreference read from JSON comes back as aBigDecimaland a:datetimeone as a time, exactly as they did from YAML; both are stored as their exact JSON strings. Only these two: every other declared type survives JSON unchanged, and running the full writer cast on read would turn a storednilinto""or0for non-nullable strings and numbers. The preference declarations are the schema; no second schema is introduced. - Symbols are not stored. Keys become strings; symbol values become strings. Code comparing preference values to symbols must compare strings.
- The conversion runs inside the 6.0 migration, not in an upgrade task.
spree:upgraderuns its manifest afterdb:migrate(seeupgrade.rakeand the CLI'supgradecommand), so a backfill scheduled there would find the YAML column already replaced. This is the reasoningConsolidateMetadataColumnsrecorded for the same problem. The proposal's "add a column, backfill with a task, swap later" order does not work with Spree's upgrade runner. - The migration never uses models. It reads rows with
select_all, parses withYAML.safe_loadand an explicit permitted-class list, and writes withexec_update. A model would load through whichever coder the current code declares, which is the thing being changed. The one thing it asks a class is which of its preferences are:password(Decision 12), from the declaration alone, without loading a record. - No batching or background job. Every table in the inventory is configuration-sized (one row per calculator, rule, gateway, store, import). The append-only log table the proposal anticipated no longer exists.
- A row still holding YAML fails when it is read, not at boot (decided 2026-10-01). A YAML document cannot be loaded as a hash, so reading that record raises, and the
spree:upgrade:preferences_jsontask finds and converts any leftovers. A dedicated error naming the task was built first and then dropped with the custom coder (see Coder below): the migration converts every row it can read and stops on any it cannot, so the error only covered rows written by pre-6.0 code after the migration ran. A boot check was rejected: boot also happens where there is no database (asset builds) and before the converting migration has run, so it would block the very migration that repairs the data, and it would cost a query per table on every start. Accepted trade-off: a missed row breaks the requests that touch that record rather than the whole application, so the failure is loud but local. - Decimals are stored as exact strings (decided 2026-10-01):
{"amount": "9.99"}, which is whatBigDecimal#as_jsonproduces and what the API already returns. The writer keeps that same string in memory (as do tier ladders), so a record just written and one read back hold identical values and setting a decimal to the value it already has is not a change; thepreferred_*reader returns theBigDecimal. A JSON number was rejected because Ruby and JavaScript parse it as a float before the read cast can restore it. Accepted trade-off: a numeric comparison in SQL needs an explicit cast ((preferences->>'amount')::numeric). - Tiered calculators store tiers as a list, not a hash with number keys (decided 2026-10-01).
Spree::Calculator::TieredPercentandTieredFlatRatechangepreference :tiers, :hashtopreference :tiers, :arrayof{ "threshold", "value" }objects, sorted by threshold. Onevaluekey for both calculators (settled during implementation, after review found the dashboard's tiers editor cannot tell which calculator it is editing): the calculator's type already says whether the value is a percentage or an amount, and per-calculator keys would have needed a new field in the preference schema. JSON object keys are always strings, so the hash shape could only survive with a hidden cast; a list reads naturally in JSON and TypeScript. This changes thetiersvalue these calculators show through the Admin API, which is a breaking change listed in the upgrade guide. The migration converts the shape. Dropping both calculators was considered and rejected (2026-10-01): they discount by spend, while price-list and catalog volume pricing works per variant by quantity, so they are not duplicates; banded promotions with the item-total rule could replace them, but only as several promotions for what is one today. - Secrets live in their own encrypted column,
secret_preferences(decided 2026-10-01). A:passwordpreference is read from and written tosecret_preferences, atextcolumn holding a JSON object and declared with Rails' standardencrypts. Everything else stays inpreferences. The declaration DSL, thepreferred_*methods, masking andPreferenceSchemaare unchanged — only where the value is stored moves. Rejected: encrypting each secret inside thepreferencesJSON (it would call Rails' encryptor by hand and rebuild key rotation, plain-text fallback and log filtering thatencryptsalready provides, and secrets would still travel whereverpreferencesis exported or logged); and a polymorphicSpree::Credentialmodel (an extra query per gateway load, preference dirty tracking that must look across a child record, secrets silently lost when a record is duplicated — a credential lifecycle nothing needs yet; it can be built later on top of this without touching declarations again). - Encryption follows Spree's existing rule: on whenever keys are configured. New projects get Active Record encryption keys at setup, so their secrets are encrypted from the first save. An installation without keys keeps working with secrets in plain text inside
secret_preferences, and gets the production boot warning Spree already logs for its other encrypted fields. This is the same conditionSpree::WebhookEndpoint,Spree::GatewayCustomerandSpree::UserIdentityuse. - Existing secrets are moved by the migration and encrypted by an upgrade task; nothing breaks in between. The migration copies each
:passwordvalue out ofpreferencesintosecret_preferencesas plain JSON — it needs no encryption keys and so can never fail for lack of them.encrypts :secret_preferences, support_unencrypted_data: truekeeps those plain values readable (the precedent isSpree::UserIdentity), so every gateway keeps working the moment the migration finishes. Thespree:upgrade:encrypt_secret_preferencestask then encrypts every row that is still plain; without it a row is encrypted on its next save anyway. A secret's reader looks inpreferencesfirst and then insecret_preferences(settled during implementation): a secret assigned as part of a whole hash waits there untilbefore_savemoves it, and a secret of a class the migration could not load stays readable untilspree:upgrade:preferences_jsonmoves it. The writer always writes the encrypted column and removes any copy frompreferences.
Design Details
Coder
No Spree-specific coder (settled during implementation). The preferences column is a native JSON column, which Rails already converts to and from a hash, so the only coder is Spree::Metadata::HashSerializer, the one metadata uses, whose sole job is to let the loaded hash answer to symbol and string keys alike. Without it, a symbol lookup on the raw hash (calculator.preferences[:amount], common in Spree and in applications) would quietly return nil instead of the value. Decimals are kept as exact strings by the preference writer, so the coder does not need to normalize them.
serialize :preferences, coder: Spree::Metadata::HashSerializer
as_json turns BigDecimal into its exact string ("9.99"), Date into ISO 8601 and symbols into strings, so dump never hands the adapter a Ruby-only type. Rails' JSON column type has already decoded the value before load sees it, on every adapter, so a string reaching load can only be a leftover YAML document.
Reads cast through the declared type
define_method preference_getter_method(name) do
stored = (preferences || {}).fetch(name) { return default.call }
convert_preference_value(stored, type, nullable: nullable)
end
convert_preference_value is idempotent for every type it handles, so casting on read changes nothing for values written in this process.
Places that assume symbol keys
Three raw reads (provision_defaults.rb, promotion/rules/country.rb twice) keep working through indifferent access. Two places compare key sets and need a stringify: Spree::Base's after_initialize backfill (default_preferences.except(*preferences.keys) would otherwise see every default as missing and dirty every record on load) and Preferable#preference_change.
Migration
One migration, ConvertPreferencesToJson, for every table in the inventory:
- For each table that has a
preferencescolumn whose values are YAML: addpreferences_json(jsonbif the adapter has it, elsejson), convert each row, droppreferences, renamepreferences_jsontopreferences. - For the two tables already typed JSON: rewrite each row whose value is a string in place.
- Drop
spree_preferences. The installation id is not copied; rolling back recreates the table empty.
Conversion per row: YAML.safe_load(text, permitted_classes: [Symbol, BigDecimal, Date, Time, ActiveSupport::TimeWithZone, ActiveSupport::TimeZone, ActiveSupport::HashWithIndifferentAccess], aliases: true) — the list Spree's engine already permits, plus Date — then as_json. Tables are discovered from the schema (any table with a preferences column), as the metadata migration does, so an application's own Preferable table converts too.
A row that does not parse — typically an old !ruby/hash-with-ivars:ActionController::Parameters dump, which the current code already cannot load — stops the migration with the table name and row id, rather than being converted to {}.
Secret preferences
# PreferableClassMethods — routing by declared type; the DSL is untouched
store_accessor(type == :password ? :secret_preferences : :preferences, name, prefix: :preferred)
# A model whose table has the column
encrypts :secret_preferences, support_unencrypted_data: true if ActiveRecord::Encryption.config.has_primary_key?
serialize :secret_preferences, coder: Spree::SecretPreferences::TextHashSerializer # JSON in a text column, then HashSerializer
- Which tables get the column:
spree_payment_methodsandspree_integrations, the only tables whose classes declare:passwordpreferences today. A model opts in by includingSpree::SecretPreferences; an application's own table adds the column itself. - Declaring a secret on a model that has not opted in raises
ArgumentErrorat declaration time, naming the concern and the column to add. A secret is never silently written to the plain column. - A secret's default is never copied into storage. The backfill that fills in missing defaults on load skips secrets; their defaults are read through the
preferred_*reader. Masking reads only the stored value, so an unset secret never reveals its default. - Gateway connection options (
Spree::Gateway#options) merge both columns and every secret's reader, so gateways still receive their keys. - An empty
secret_preferencescolumn reads asnil, not{}. Encrypted attributes compare the stored value with the loaded one, so an empty hash would mark every record without secrets as changed the moment a secret was read. - Declarations stay current down the class tree. Each class keeps only its own declarations and merges its ancestors' on demand, cached until the next declaration anywhere — so a secret a decorator adds to
Spree::PaymentMethodafter its subclasses loaded is still routed to the encrypted column. - Code that writes preferences without callbacks writes both columns.
update_columns(preferences: ...)skips the save hook that moves secrets, so a writer like Stripe'smigrate_stripe_webhook_keyssets values through thepreferred_*writers and then writespreferencesandsecret_preferencestogether. - The safety-net task converts a column that is still
text— a table whose model the migration did not load — with the same steps as the migration, instead of pointing back at a migration that has already run. - Each preference is a Rails store accessor (decided 2026-10-01, at the author's request, in the same pull request).
preferenceregisters a store accessor onpreferences(orsecret_preferencesfor a secret) with thepreferredprefix, so Rails supplies the change methods; the hand-written ones are gone, and their after-save forms take Rails' names (saved_change_to_preferred_<name>?,preferred_<name>_before_last_save). One table of definitions per class (preference_definitions) replaces the five helper methods every preference generated and themethods.grepthat found them. The rawpreferenceshash still holds every default (the author's requirement), so the backfill on load stays.activerecord-typedstorewas considered and rejected: it wants all fields declared in one block per column, fixes defaults at class load, and would still need Spree's layer forparse_on_set, nullable values, choices and deprecations. - Rows of an uninstalled class are skipped by both upgrade tasks, with a line naming the class, so one leftover gateway from a removed gem cannot stop the run before the other rows. Their secrets stay readable where they are.
- Change detection across both columns:
previously_changed_preference_nameslists what the last save changed in either column. Stripe uses it to recognise a save that only stored webhook registration results. preferencesis the only column the Admin API, exports and event payloads read. Secrets reach the API only through the existing masking, which now reads throughpreferred_*rather than the raw hash.- Copying a record (
dup) carriessecret_preferencesalong withpreferences, so duplicating a payment method keeps its keys. - Which keys are secrets comes from the class declarations (
declared_preference_types, recorded per class when a preference is declared, so no record is built). The migration resolves each row's class from itstypecolumn; a row whose class cannot be loaded (an uninstalled gem) keeps its secrets inpreferencesand is reported, and the safety-net task moves them once the class is back.
Tiered calculators
# Before: { 100.0 => 10.0, 250.0 => 15.0 }
preference :tiers, :array, default: []
# After: [{ "threshold" => "100.0", "value" => "10.0" }, { "threshold" => "250.0", "value" => "15.0" }]
compute picks the last tier whose threshold the amount reaches. Validation checks that each entry has a positive threshold and, for the percent calculator, a rate from 0 to 100, and that thresholds are unique. The migration converts the hash for rows whose type is either calculator or an application's subclass of one, keeping the decimal-string rule above. The dashboard's tiers editor (preferences-form.tsx in @spree/dashboard-core) reads and writes the list.
Afterwards
yaml_column_permitted_classes injection is removed from Spree::Core::Engine and from the dummy app generator. The comments in preferable.rb and base.rb that explain YAML behaviour go with it.
Migration Path
- Change the tiered calculators to the list shape (code and specs).
- Switch
Preferableto JSON throughSpree::Metadata::HashSerializer; restore decimals on read; stringify the two key-set comparisons; delete the deadserializeonSpree::Address. - Add the
secret_preferencescolumn and routing for:passwordpreferences; raise when a secret is declared on a table without it; read masking through thepreferred_*methods. - Add the
ConvertPreferencesToJsonmigration, including the tiers reshaping and moving secrets intosecret_preferences, with a reversibledown(JSON back to YAML restores every value through the declared types; tiers convert back to the hash). - Add two steps to the 5.6→6.0 manifest:
spree:upgrade:preferences_json(safety net: reports and converts any row still holding YAML or a secret left inpreferences) andspree:upgrade:encrypt_secret_preferences(encrypts every plainsecret_preferencesrow; reports and skips when no keys are configured). Both idempotent. - Remove the YAML permitted-classes configuration,
Spree::Preference,Spree::Preferences::StoreandSpree::Preferences::ScopedStore. - Document the change in
docs/developer/upgrades/5.6-to-6.0.mdx(including that gateway secrets are now encrypted when keys are set, and linking the existing encryption-keys section), list gateway secrets among encrypted values indocs/developer/deployment/environment_variables.mdx, and updatedocs/developer/customization/model-preferences.mdx(keys are strings in the raw hash; values come back typed through thepreferred_*readers). (docs/developer/core-concepts/promotions.mdxnames the tiered calculators but shows no tiers, so it needed no change.) - Specs: a coder round trip for every declared type, secrets landing only in
secret_preferences(encrypted with keys, plain without), a moved plain secret staying readable, the encrypt task, the migration on each adapter, the existing preference specs passing unchanged, and the tiered calculator specs moved to the list shape.
Constraints on Current Work
- New
preferencescolumns are JSON. Use thet.respond_to?(:jsonb)guard from the root CLAUDE.md. Do not add anothertextpreferences column. - Read preferences through
preferred_*orget_preference, not the raw hash. The raw hash is where YAML and JSON differ; the readers hide it. - Do not store symbols,
BigDecimalorTimeobjects in a preference you expect back unchanged without a declared type that casts it. - Every secret is a
:passwordpreference. That type is what sends a value to the encrypted column. A secret declared:string, or nested inside a:hashor:arraypreference, stays in plain text. - New gateway or integration tables need a
secret_preferencestextcolumn if their classes declare:passwordpreferences. - Never write
preferenceswithupdate_columns/update_allwhen it may hold a secret. Those skip the hook that moves secrets to the encrypted column. Set values through thepreferred_*writers and write bothpreferencesandsecret_preferences. - No number keys in
:hashpreferences. JSON turns them into strings. Use a list of objects, as the tiered calculators now do. - Do not query
preferencesin SQL until this lands. On the two JSON-typed tables the value is a YAML string today and any JSON operator silently matches nothing.
Open Questions
None blocking. Deferred:
- A credential lifecycle (rotation history, last used, expiry, its own permission). Nothing needs it yet; it can be built on
secret_preferenceswithout changing declarations. - Dropping
spree_gateways, the unused 4.3 table. Unblocked by confirming no extension still reads it; a candidate for 6.1 cleanup.
References
docs/plans/decisions.md— 2026-08-09 metadata consolidation (the migration precedent) and the Stripe port entry naming encryption ofpreferencesas a live questiondocs/plans/6.0-store-scoped-configuration.md— store preferences and the backfill-marker wrinkledocs/developer/customization/model-preferences.mdxdocs/developer/deployment/environment_variables.mdx— encryption keys, enabling encryption on an existing installationspree/core/app/models/spree/user_identity.rb— the plain-text-then-encrypt precedentspree/core/lib/spree/core/preferences/preferable.rb,preferable_class_methods.rbspree/core/db/migrate/20260809120001_consolidate_metadata_columns.rb