Skip to content

Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.

Re-sign rich text attachment nodes without the previous secret_key_base

Guido Boykeabd2d732b306
diff --git a/app/models/concerns/rich_text_embed_retention.rb b/app/models/concerns/rich_text_embed_retention.rb
index 1ecf279..e318575 100644
--- a/app/models/concerns/rich_text_embed_retention.rb
+++ b/app/models/concerns/rich_text_embed_retention.rb
@@ -8,8 +8,8 @@
# (rich_text.rb: `self.embeds = body.attachables.grep(ActiveStorage::Blob).uniq`). A sgid
# stops resolving for everything stored before a secret_key_base rotation, and then
# editing only the words of a text detached every image and enqueued the purge of its
-# file. Which embed a dead node meant cannot be told without
-# the old key, so every stored embed that would be dropped is kept. The price is that an
+# file. A dead node is not trusted to say which embed it meant, so every stored embed
+# that would be dropped is kept. The price is that an
# image removed in the same edit stays attached until the text is saved again with
# readable nodes; RichTextSgidResign makes them readable.
#
diff --git a/app/services/rich_text_sgid_resign.rb b/app/services/rich_text_sgid_resign.rb
index 448aa3f..4c63f29 100644
--- a/app/services/rich_text_sgid_resign.rb
+++ b/app/services/rich_text_sgid_resign.rb
@@ -1,38 +1,38 @@
# frozen_string_literal: true
-# One-off repair after a secret_key_base rotation: every
-# <action-text-attachment sgid> stored before it was signed with the old key, so Rails
-# renders each image and wiki mention as "☒". This re-signs those nodes with the current key.
+# One-off repair after a secret_key_base rotation: every <action-text-attachment sgid>
+# stored before it was signed with the old key, so Rails renders each image and wiki
+# mention as "☒". This re-signs those nodes with the current key, without the old one.
#
-# The old key is an argument of this one run, never configuration: a key is rotated because
-# others may hold it, and a sgid forged with it must not gain a target. So a node is re-signed only when
-# 1. the old key verifies it (an unverified sgid is never decoded: older ones are Marshal),
-# 2. its record still exists, and
-# 3. the record belongs to this text: a blob only when it is attached to THIS rich text
-# (its embeds), anything else (a mention) only within the carrier's organisation.
-# Everything else is counted (unreadable) or listed with its ids (gone, refused); a gone
-# blob was purged after the rotation and can only come back from the attachment backup.
+# A stale signature proves nothing, so the node itself is never trusted. A node is
+# re-signed only when what the app stored NEXT TO the text, while the key was still
+# valid, names the same target:
+# * an image: a blob attached to THIS rich text (its embeds);
+# * a wiki mention (notes only): a target among the note's outgoing references.
+# The target is read from the sgid's payload when that payload is JSON (never Marshal:
+# older sgids are, and an unverified Marshal payload is not to be loaded); an image node
+# whose payload cannot be read is matched to an embed by its filename and filesize.
+# Everything else is listed with its ids: a blob that is no longer attached was purged
+# after the rotation and can only come back from the attachment backup.
#
# Writes with update_column like RichTextEmbedUrlSweep: no validation, no NoteVersion, no
# embeds sync; a note's derived markdown_body is re-derived so the next edit shows no
# spurious diff. Idempotent: a node that resolves is left alone.
#
-# Entry point: RichTextSgidResign.call(old_secret_key_base:, dry_run: true)
+# Entry point: RichTextSgidResign.call(dry_run: true)
class RichTextSgidResign
- Result = Struct.new(:scanned, :changed, :resigned, :unreadable, :gone, :refused, :dry_run, keyword_init: true)
+ Result = Struct.new(:scanned, :changed, :resigned, :unmatched, :dry_run, keyword_init: true)
- def self.call(old_secret_key_base:, dry_run: true) = new(old_secret_key_base: old_secret_key_base, dry_run: dry_run).call
+ BLOB_MODEL = 'ActiveStorage::Blob'
- def initialize(old_secret_key_base:, dry_run: true)
- raise ArgumentError, 'old_secret_key_base is required' if old_secret_key_base.blank?
+ def self.call(dry_run: true) = new(dry_run: dry_run).call
- # The same derivation GlobalID's railtie uses for SignedGlobalID.verifier.
- @old_verifier = GlobalID::Verifier.new(Rails.application.key_generator(old_secret_key_base).generate_key('signed_global_ids'))
+ def initialize(dry_run: true)
@dry_run = dry_run
end
def call
- @result = Result.new(scanned: 0, changed: 0, resigned: 0, unreadable: 0, gone: [], refused: [], dry_run: @dry_run)
+ @result = Result.new(scanned: 0, changed: 0, resigned: 0, unmatched: [], dry_run: @dry_run)
ActsAsTenant.without_tenant do
candidates.find_each do |rich_text|
@@ -51,68 +51,88 @@ class RichTextSgidResign
private
def candidates
- ActionText::RichText.where('body LIKE ?', "%<#{ActionText::Attachment.tag_name}%").includes(:record)
+ ActionText::RichText.where('body LIKE ?', "%<#{ActionText::Attachment.tag_name}%")
+ .includes(:record, embeds_attachments: :blob)
end
# @return [String, nil] the html with re-signed nodes, or nil when nothing changed
def resign(rich_text)
fragment = ActionText::Fragment.wrap(rich_text.body.to_html)
- touched = false
- fragment.find_all(ActionText::Attachment.tag_name).each do |node|
- sgid = resign_stale_node(rich_text, node['sgid'])
- next if sgid.nil?
+ stale = fragment.find_all(ActionText::Attachment.tag_name).reject { |node| current?(node['sgid']) }
+ return if stale.empty?
- node['sgid'] = sgid
- touched = true
- @result.resigned += 1
- end
- touched ? fragment.to_html : nil
- end
+ claimed = []
+ stale.each do |node|
+ target = stored_target(rich_text, node, claimed)
+ next if target.nil?
- # @return [String, nil] the new sgid, or nil when the node stays as it is
- def resign_stale_node(rich_text, sgid)
- return if sgid.blank? || signed_with_current_key?(sgid)
-
- gid = SignedGlobalID.parse(sgid, for: ActionText::Attachable::LOCATOR_NAME, verifier: @old_verifier)
- if gid.nil?
- @result.unreadable += 1
- return
+ claimed << target
+ node['sgid'] = target.attachable_sgid
+ @result.resigned += 1
end
-
- record = locate(gid)
- return list(:gone, rich_text, gid) if record.nil?
- return list(:refused, rich_text, gid) unless belongs_to_text?(rich_text, record)
-
- record.attachable_sgid
+ claimed.any? ? fragment.to_html : nil
end
# A sgid of the current key whose record is gone raises instead of answering nil; it is
# still current, and no re-signing would bring its record back.
- def signed_with_current_key?(sgid)
+ def current?(sgid)
+ return true if sgid.blank?
+
GlobalID::Locator.locate_signed(sgid, for: ActionText::Attachable::LOCATOR_NAME).present?
rescue ActiveRecord::RecordNotFound
true
end
- def locate(gid)
- GlobalID::Locator.locate(gid)
- rescue ActiveRecord::RecordNotFound
+ def stored_target(rich_text, node, claimed)
+ gid = payload_gid(node['sgid'])
+ target, reason =
+ if gid.nil? then [embed_by_attributes(rich_text, node, claimed), :unreadable]
+ elsif gid.model_name == BLOB_MODEL then [embeds_of(rich_text).find { |blob| blob.id == gid.model_id }, :not_attached]
+ else [referenced_target(rich_text.record, gid), :no_reference]
+ end
+ target || unmatched(rich_text, node, gid, reason)
+ end
+
+ # The GlobalID a sgid names, read from its payload WITHOUT verifying it. JSON only: the
+ # current envelope ({"_rails": {"data", "pur"}}) and the older self-describing one
+ # ({"gid", "purpose"}). Anything else (Marshal) answers nil and is never loaded.
+ def payload_gid(sgid)
+ message = JSON.parse(Base64.urlsafe_decode64(sgid.to_s.split('--').first.to_s))
+ return unless message.is_a?(Hash)
+
+ envelope = message['_rails'].is_a?(Hash) ? message['_rails'] : message
+ uri, purpose = envelope.values_at('data', 'pur')
+ uri, purpose = envelope.values_at('gid', 'purpose') if uri.nil?
+ return unless purpose.to_s == ActionText::Attachable::LOCATOR_NAME
+
+ GlobalID.parse(uri)
+ rescue ArgumentError, JSON::ParserError
nil
end
- def belongs_to_text?(rich_text, record)
- if record.is_a?(ActiveStorage::Blob)
- ActiveStorage::Attachment.exists?(record: rich_text, name: 'embeds', blob: record)
- else
- carrier_organisation_id = rich_text.record.try(:organisation_id)
- carrier_organisation_id.present? && record.try(:organisation_id) == carrier_organisation_id
+ def embeds_of(rich_text) = rich_text.embeds_attachments.map(&:blob)
+
+ # An image node whose payload is not readable: the one embed with its filename and
+ # filesize that no other node of the text has taken. Two candidates are no answer.
+ def embed_by_attributes(rich_text, node, claimed)
+ return if node['filename'].blank? || node['filesize'].blank?
+
+ matches = (embeds_of(rich_text) - claimed).select do |blob|
+ blob.filename.to_s == node['filename'] && blob.byte_size.to_s == node['filesize']
end
+ matches.one? ? matches.first : nil
+ end
+
+ def referenced_target(carrier, gid)
+ return unless carrier.respond_to?(:outgoing_references)
+
+ carrier.outgoing_references.find { |ref| ref.target_type == gid.model_name && ref.target_id == gid.model_id }&.target
end
- # Model and id, never gid.to_s: on a SignedGlobalID that signs the id again.
- def list(kind, rich_text, gid)
- @result[kind] << { rich_text_id: rich_text.id, record_type: rich_text.record_type, record_id: rich_text.record_id,
- target_type: gid.model_name, target_id: gid.model_id }
+ def unmatched(rich_text, node, gid, reason)
+ @result.unmatched << { rich_text_id: rich_text.id, record_type: rich_text.record_type, record_id: rich_text.record_id,
+ reason: reason, target_type: gid&.model_name, target_id: gid&.model_id,
+ filename: node['filename'] }
nil
end
diff --git a/lib/tasks/rich_text_embeds.rake b/lib/tasks/rich_text_embeds.rake
index ac3d000..7ac1c51 100644
--- a/lib/tasks/rich_text_embeds.rake
+++ b/lib/tasks/rich_text_embeds.rake
@@ -12,25 +12,20 @@ namespace :rich_text_embeds do
puts " #{result.dry_run ? 'Würden geändert' : 'Geändert'}: #{result.changed}"
end
- # After a secret_key_base rotation (RichTextSgidResign). The old key comes from the
- # environment of this one run only; read it with `read -rs OLD_SECRET_KEY_BASE` and
- # export it, so it lands neither in the shell history nor in a file.
- desc 'Attachment-Knoten mit dem aktuellen Schlüssel neu signieren. OLD_SECRET_KEY_BASE setzen, zuerst DRY_RUN=1.'
+ # After a secret_key_base rotation (RichTextSgidResign): re-signs the attachment nodes
+ # whose target the app stored next to the text. Needs no old key.
+ desc 'Attachment-Knoten nach einem Tausch von secret_key_base neu signieren. Immer zuerst mit DRY_RUN=1.'
task resign_sgids: :environment do
- old_secret_key_base = ENV.fetch('OLD_SECRET_KEY_BASE', '')
- abort 'OLD_SECRET_KEY_BASE fehlt (der Schlüssel VOR dem Tausch).' if old_secret_key_base.blank?
- abort 'OLD_SECRET_KEY_BASE ist der aktuelle Schlüssel.' if old_secret_key_base == Rails.application.secret_key_base
-
- result = RichTextSgidResign.call(old_secret_key_base: old_secret_key_base, dry_run: ENV['DRY_RUN'] == '1')
+ result = RichTextSgidResign.call(dry_run: ENV['DRY_RUN'] == '1')
puts format('rich_text_embeds:resign_sgids%<mode>s', mode: result.dry_run ? ' (DRY_RUN, nichts geschrieben)' : '')
puts " Rich-Texte mit Attachment-Knoten: #{result.scanned}"
puts " #{result.dry_run ? 'Würden geändert' : 'Geändert'}: #{result.changed} Texte, #{result.resigned} Knoten"
- puts " Mit keinem der beiden Schlüssel lesbar: #{result.unreadable}"
- { gone: 'Ziel fehlt (Blob gelöscht, aus der Sicherung holen)', refused: 'Abgelehnt (gehört nicht zu diesem Text)' }.each do |kind, label|
- rows = result[kind]
- puts " #{label}: #{rows.size}"
+ puts " Nicht zugeordnet: #{result.unmatched.size}"
+ result.unmatched.group_by { |row| row[:reason] }.each do |reason, rows|
+ puts " #{reason}: #{rows.size}"
rows.each do |row|
- puts " #{row[:record_type]} #{row[:record_id]} (Rich-Text #{row[:rich_text_id]}) -> #{row[:target_type]} #{row[:target_id]}"
+ target = row[:target_id] ? "#{row[:target_type]} #{row[:target_id]}" : "Datei #{row[:filename].inspect}"
+ puts " #{row[:record_type]} #{row[:record_id]} (Rich-Text #{row[:rich_text_id]}) -> #{target}"
end
end
end
diff --git a/spec/services/rich_text_sgid_resign_spec.rb b/spec/services/rich_text_sgid_resign_spec.rb
index 849246d..f601e85 100644
--- a/spec/services/rich_text_sgid_resign_spec.rb
+++ b/spec/services/rich_text_sgid_resign_spec.rb
@@ -3,17 +3,16 @@
require 'rails_helper'
# What: RichTextSgidResign re-signs the attachment nodes of stored rich texts whose sgid
-# was signed with the previous secret_key_base.
+# was signed with a previous secret_key_base, without that key.
# Type: Service spec (2026-10-20).
# Why: A secret_key_base rotation turns every stored image and wiki mention into a
-# "☒". The old key comes only from the one run of the task, so a node is accepted
-# only when the old key signed it AND its target belongs to this text: an image
-# only when it is attached to THIS text, a mention only within the text's own
-# organisation. Others may hold a rotated key; a sgid forged with it must gain nothing.
+# "☒". A stale signature proves nothing, so a node is re-signed only when what the
+# app stored next to the text names the same target: an embed of this text for an
+# image, an outgoing reference of the note for a mention. A forged node, or one
+# that names anything else, must gain nothing. A Marshal payload is never loaded.
RSpec.describe RichTextSgidResign, :ci_guard do
let(:organisation) { create(:organisation) }
let(:project) { create(:project, organisation: organisation) }
- let(:old_secret_key_base) { Rails.application.secret_key_base }
before { ActsAsTenant.current_tenant = organisation }
@@ -38,14 +37,34 @@ RSpec.describe RichTextSgidResign, :ci_guard do
def mention(note) = node(note, ' content-type="application/vnd.actiontext.mention"')
- # What a rotation does: the sgids written so far were signed with the key
- # the application booted with, every one minted from now on uses the new key.
+ # What a rotation does: the sgids written so far were signed with the key the
+ # application booted with, every one minted from now on uses the new key.
def rotate_secret_key_base!
- key = ActiveSupport::KeyGenerator.new('a-rotated-secret-key-base' * 4, iterations: 1000)
- .generate_key('signed_global_ids')
- SignedGlobalID.verifier = GlobalID::Verifier.new(key)
+ SignedGlobalID.verifier = GlobalID::Verifier.new(key_for('a-rotated-secret-key-base'))
end
+ def key_for(secret) = ActiveSupport::KeyGenerator.new(secret * 4, iterations: 1000).generate_key('signed_global_ids')
+
+ # The self-describing metadata of older globalid releases, serialised with Marshal.
+ def marshal_sgid(record)
+ GlobalID::Verifier.new(key_for('an-old-secret-key-base'), serializer: Marshal)
+ .generate({ 'gid' => record.to_gid.to_s, 'purpose' => 'attachable', 'expires_at' => nil })
+ end
+
+ # A payload anybody can write: the current envelope with a made-up digest.
+ def unsigned_sgid(record)
+ payload = { _rails: { data: record.to_gid.to_s, pur: 'attachable' } }.to_json
+ "#{Base64.urlsafe_encode64(payload)}--#{'0' * 40}"
+ end
+
+ def ticket_with_body(html)
+ create(:ticket, project: project, organisation: organisation, content: '<div>x</div>').tap do |ticket|
+ ticket.content.update_column(:body, html)
+ end
+ end
+
+ def attach_embed(rich_text, blob) = ActiveStorage::Attachment.create!(record: rich_text, name: 'embeds', blob: blob)
+
def sgids_of(record)
Nokogiri::HTML.fragment(record.reload.content.body.to_html).css('action-text-attachment').map { |n| n['sgid'] }
end
@@ -56,7 +75,7 @@ RSpec.describe RichTextSgidResign, :ci_guard do
false
end
- def run(dry_run: false) = described_class.call(old_secret_key_base: old_secret_key_base, dry_run: dry_run)
+ def run(dry_run: false) = described_class.call(dry_run: dry_run)
it 'brings back an image of the text and renders it instead of the glyph' do
blob = png_blob
@@ -64,9 +83,7 @@ RSpec.describe RichTextSgidResign, :ci_guard do
rotate_secret_key_base!
expect(Ticket.find(ticket.id).content.to_s).to include('☒')
- result = run
-
- expect(result.resigned).to eq(1)
+ expect(run.resigned).to eq(1)
expect(sgids_of(ticket)).to all(satisfy { |sgid| resolves?(sgid) })
expect(Ticket.find(ticket.id).content.to_s).not_to include('☒')
end
@@ -76,9 +93,7 @@ RSpec.describe RichTextSgidResign, :ci_guard do
stale = sgids_of(ticket)
rotate_secret_key_base!
- result = run(dry_run: true)
-
- expect(result.to_h).to include(changed: 1, resigned: 1, dry_run: true)
+ expect(run(dry_run: true).to_h).to include(changed: 1, resigned: 1, dry_run: true)
expect(sgids_of(ticket)).to eq(stale)
end
@@ -90,7 +105,7 @@ RSpec.describe RichTextSgidResign, :ci_guard do
expect(run.to_h).to include(changed: 0, resigned: 0)
end
- it 'brings back a wiki mention within the organisation and re-derives the markdown body' do
+ it 'brings back a wiki mention the note references and re-derives the markdown body' do
target = create(:note, project: project, organisation: organisation, title: 'Zielnotiz')
note = create(:note, project: project, organisation: organisation, content: "<div>Siehe</div>#{mention(target)}")
rotate_secret_key_base!
@@ -101,45 +116,61 @@ RSpec.describe RichTextSgidResign, :ci_guard do
expect(note.reload.markdown_body).to eq(Wiki::MarkdownSerializer.new.to_markdown(note.content.body.to_html))
end
- it 'refuses an image that is not attached to this text' do
- foreign = png_blob('fremd.png')
- ticket = create(:ticket, project: project, organisation: organisation, content: '<div>x</div>')
- ticket.content.update_column(:body, "<div>x</div>#{node(foreign)}")
+ it 'refuses a mention the note does not reference, even within the organisation' do
+ target = create(:note, project: project, organisation: organisation, title: 'Nie verlinkt')
+ note = create(:note, project: project, organisation: organisation, content: '<div>x</div>')
+ note.content.update_column(:body, "<div>x</div>#{mention(target)}")
rotate_secret_key_base!
result = run
expect(result.resigned).to eq(0)
- expect(result.refused).to contain_exactly(hash_including(rich_text_id: ticket.content.id, target_type: 'ActiveStorage::Blob', target_id: foreign.id))
- expect(resolves?(sgids_of(ticket).first)).to be(false)
+ expect(result.unmatched).to contain_exactly(hash_including(reason: :no_reference, target_type: 'Note', target_id: target.id))
end
- it 'refuses a mention of a record in another organisation' do
- other = create(:organisation)
- target = ActsAsTenant.with_tenant(other) do
- create(:note, project: create(:project, organisation: other), organisation: other, title: 'Fremd')
- end
- note = create(:note, project: project, organisation: organisation, content: '<div>x</div>')
- note.content.update_column(:body, "<div>x</div>#{mention(target)}")
+ it 'refuses an image that is not attached to this text' do
+ foreign = png_blob('fremd.png')
+ ticket = ticket_with_body("<div>x</div>#{node(foreign)}")
rotate_secret_key_base!
result = run
expect(result.resigned).to eq(0)
- expect(result.refused).to contain_exactly(hash_including(target_type: 'Note', target_id: target.id))
+ expect(result.unmatched).to contain_exactly(
+ hash_including(rich_text_id: ticket.content.id, reason: :not_attached, target_type: 'ActiveStorage::Blob', target_id: foreign.id)
+ )
+ expect(resolves?(sgids_of(ticket).first)).to be(false)
end
- it 'refuses a sgid the old key did not sign, without decoding it' do
- blob = png_blob
- ticket = create(:ticket, project: project, organisation: organisation, content: "<div>x</div>#{node(blob)}")
- rotate_secret_key_base!
-
- result = described_class.call(old_secret_key_base: 'not-the-old-key' * 8, dry_run: false)
+ it 'gives a forged, unsigned node nothing it could not already show' do
+ foreign = png_blob('fremd.png')
+ ticket = ticket_with_body(%(<div>x</div><action-text-attachment sgid="#{unsigned_sgid(foreign)}"></action-text-attachment>))
- expect(result.to_h).to include(resigned: 0, unreadable: 1)
+ expect(run.resigned).to eq(0)
expect(resolves?(sgids_of(ticket).first)).to be(false)
end
+ it 'matches a Marshal node to its embed by filename and filesize, without loading the payload' do
+ blob = png_blob('alt.png')
+ ticket = ticket_with_body(%(<div>x</div><action-text-attachment sgid="#{marshal_sgid(blob)}" filename="alt.png" filesize="#{blob.byte_size}"></action-text-attachment>))
+ attach_embed(ticket.content, blob)
+ expect(Marshal).not_to receive(:load)
+
+ expect(run.resigned).to eq(1)
+ expect(ActionText::Attachable.from_attachable_sgid(sgids_of(ticket).first)).to eq(blob)
+ end
+
+ it 'lists a Marshal node it cannot match' do
+ blob = png_blob('alt.png')
+ ticket = ticket_with_body(%(<div>x</div><action-text-attachment sgid="#{marshal_sgid(blob)}" filename="anders.png" filesize="1"></action-text-attachment>))
+ attach_embed(ticket.content, blob)
+
+ result = run
+
+ expect(result.resigned).to eq(0)
+ expect(result.unmatched).to contain_exactly(hash_including(reason: :unreadable, filename: 'anders.png', target_id: nil))
+ end
+
it 'lists an image whose blob is gone so it can be restored from the backup' do
blob = png_blob
ticket = create(:ticket, project: project, organisation: organisation, content: "<div>x</div>#{node(blob)}")
@@ -150,10 +181,8 @@ RSpec.describe RichTextSgidResign, :ci_guard do
result = run
expect(result.resigned).to eq(0)
- expect(result.gone).to contain_exactly(hash_including(rich_text_id: ticket.content.id, target_type: 'ActiveStorage::Blob', target_id: blob.id))
- end
-
- it 'refuses to run without the old key' do
- expect { described_class.call(old_secret_key_base: '', dry_run: true) }.to raise_error(ArgumentError)
+ expect(result.unmatched).to contain_exactly(
+ hash_including(rich_text_id: ticket.content.id, reason: :not_attached, target_type: 'ActiveStorage::Blob', target_id: blob.id)
+ )
end
end