Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.
Ruby agplai-assistantdockerdsgvogdprhotwireissue-trackerjira-alternativeknowledge-basellmmcp-servermeeting-notesopen-sourceproject-managementredmine-alternativeruby-on-railsself-hostedticketingtime-tracking
Re-sign rich text attachment nodes without the previous secret_key_base
Guido Boykeabd2d732b306
diff --git a/app/models/concerns/rich_text_embed_retention.rb b/app/models/concerns/rich_text_embed_retention.rbindex 1ecf279..e318575 100644--- a/app/models/concerns/rich_text_embed_retention.rb+++ b/app/models/concerns/rich_text_embed_retention.rb@@ -8,8 +8,8 @@# (rich_text.rb: `self.embeds = body.attachables.grep(ActiveStorage::Blob).uniq`). A sgid# stops resolving for everything stored before a secret_key_base rotation, and then# editing only the words of a text detached every image and enqueued the purge of its-# file. Which embed a dead node meant cannot be told without-# the old key, so every stored embed that would be dropped is kept. The price is that an+# file. A dead node is not trusted to say which embed it meant, so every stored embed+# that would be dropped is kept. The price is that an# image removed in the same edit stays attached until the text is saved again with# readable nodes; RichTextSgidResign makes them readable.#diff --git a/app/services/rich_text_sgid_resign.rb b/app/services/rich_text_sgid_resign.rbindex 448aa3f..4c63f29 100644--- a/app/services/rich_text_sgid_resign.rb+++ b/app/services/rich_text_sgid_resign.rb@@ -1,38 +1,38 @@# frozen_string_literal: true-# One-off repair after a secret_key_base rotation: every-# <action-text-attachment sgid> stored before it was signed with the old key, so Rails-# renders each image and wiki mention as "☒". This re-signs those nodes with the current key.+# One-off repair after a secret_key_base rotation: every <action-text-attachment sgid>+# stored before it was signed with the old key, so Rails renders each image and wiki+# mention as "☒". This re-signs those nodes with the current key, without the old one.#-# The old key is an argument of this one run, never configuration: a key is rotated because-# others may hold it, and a sgid forged with it must not gain a target. So a node is re-signed only when-# 1. the old key verifies it (an unverified sgid is never decoded: older ones are Marshal),-# 2. its record still exists, and-# 3. the record belongs to this text: a blob only when it is attached to THIS rich text-# (its embeds), anything else (a mention) only within the carrier's organisation.-# Everything else is counted (unreadable) or listed with its ids (gone, refused); a gone-# blob was purged after the rotation and can only come back from the attachment backup.+# A stale signature proves nothing, so the node itself is never trusted. A node is+# re-signed only when what the app stored NEXT TO the text, while the key was still+# valid, names the same target:+# * an image: a blob attached to THIS rich text (its embeds);+# * a wiki mention (notes only): a target among the note's outgoing references.+# The target is read from the sgid's payload when that payload is JSON (never Marshal:+# older sgids are, and an unverified Marshal payload is not to be loaded); an image node+# whose payload cannot be read is matched to an embed by its filename and filesize.+# Everything else is listed with its ids: a blob that is no longer attached was purged+# after the rotation and can only come back from the attachment backup.## Writes with update_column like RichTextEmbedUrlSweep: no validation, no NoteVersion, no# embeds sync; a note's derived markdown_body is re-derived so the next edit shows no# spurious diff. Idempotent: a node that resolves is left alone.#-# Entry point: RichTextSgidResign.call(old_secret_key_base:, dry_run: true)+# Entry point: RichTextSgidResign.call(dry_run: true)class RichTextSgidResign- Result = Struct.new(:scanned, :changed, :resigned, :unreadable, :gone, :refused, :dry_run, keyword_init: true)+ Result = Struct.new(:scanned, :changed, :resigned, :unmatched, :dry_run, keyword_init: true)- def self.call(old_secret_key_base:, dry_run: true) = new(old_secret_key_base: old_secret_key_base, dry_run: dry_run).call+ BLOB_MODEL = 'ActiveStorage::Blob'- def initialize(old_secret_key_base:, dry_run: true)- raise ArgumentError, 'old_secret_key_base is required' if old_secret_key_base.blank?+ def self.call(dry_run: true) = new(dry_run: dry_run).call- # The same derivation GlobalID's railtie uses for SignedGlobalID.verifier.- @old_verifier = GlobalID::Verifier.new(Rails.application.key_generator(old_secret_key_base).generate_key('signed_global_ids'))+ def initialize(dry_run: true)@dry_run = dry_runenddef call- @result = Result.new(scanned: 0, changed: 0, resigned: 0, unreadable: 0, gone: [], refused: [], dry_run: @dry_run)+ @result = Result.new(scanned: 0, changed: 0, resigned: 0, unmatched: [], dry_run: @dry_run)ActsAsTenant.without_tenant docandidates.find_each do |rich_text|@@ -51,68 +51,88 @@ class RichTextSgidResignprivatedef candidates- ActionText::RichText.where('body LIKE ?', "%<#{ActionText::Attachment.tag_name}%").includes(:record)+ ActionText::RichText.where('body LIKE ?', "%<#{ActionText::Attachment.tag_name}%")+ .includes(:record, embeds_attachments: :blob)end# @return [String, nil] the html with re-signed nodes, or nil when nothing changeddef resign(rich_text)fragment = ActionText::Fragment.wrap(rich_text.body.to_html)- touched = false- fragment.find_all(ActionText::Attachment.tag_name).each do |node|- sgid = resign_stale_node(rich_text, node['sgid'])- next if sgid.nil?+ stale = fragment.find_all(ActionText::Attachment.tag_name).reject { |node| current?(node['sgid']) }+ return if stale.empty?- node['sgid'] = sgid- touched = true- @result.resigned += 1- end- touched ? fragment.to_html : nil- end+ claimed = []+ stale.each do |node|+ target = stored_target(rich_text, node, claimed)+ next if target.nil?- # @return [String, nil] the new sgid, or nil when the node stays as it is- def resign_stale_node(rich_text, sgid)- return if sgid.blank? || signed_with_current_key?(sgid)-- gid = SignedGlobalID.parse(sgid, for: ActionText::Attachable::LOCATOR_NAME, verifier: @old_verifier)- if gid.nil?- @result.unreadable += 1- return+ claimed << target+ node['sgid'] = target.attachable_sgid+ @result.resigned += 1end-- record = locate(gid)- return list(:gone, rich_text, gid) if record.nil?- return list(:refused, rich_text, gid) unless belongs_to_text?(rich_text, record)-- record.attachable_sgid+ claimed.any? ? fragment.to_html : nilend# A sgid of the current key whose record is gone raises instead of answering nil; it is# still current, and no re-signing would bring its record back.- def signed_with_current_key?(sgid)+ def current?(sgid)+ return true if sgid.blank?+GlobalID::Locator.locate_signed(sgid, for: ActionText::Attachable::LOCATOR_NAME).present?rescue ActiveRecord::RecordNotFoundtrueend- def locate(gid)- GlobalID::Locator.locate(gid)- rescue ActiveRecord::RecordNotFound+ def stored_target(rich_text, node, claimed)+ gid = payload_gid(node['sgid'])+ target, reason =+ if gid.nil? then [embed_by_attributes(rich_text, node, claimed), :unreadable]+ elsif gid.model_name == BLOB_MODEL then [embeds_of(rich_text).find { |blob| blob.id == gid.model_id }, :not_attached]+ else [referenced_target(rich_text.record, gid), :no_reference]+ end+ target || unmatched(rich_text, node, gid, reason)+ end++ # The GlobalID a sgid names, read from its payload WITHOUT verifying it. JSON only: the+ # current envelope ({"_rails": {"data", "pur"}}) and the older self-describing one+ # ({"gid", "purpose"}). Anything else (Marshal) answers nil and is never loaded.+ def payload_gid(sgid)+ message = JSON.parse(Base64.urlsafe_decode64(sgid.to_s.split('--').first.to_s))+ return unless message.is_a?(Hash)++ envelope = message['_rails'].is_a?(Hash) ? message['_rails'] : message+ uri, purpose = envelope.values_at('data', 'pur')+ uri, purpose = envelope.values_at('gid', 'purpose') if uri.nil?+ return unless purpose.to_s == ActionText::Attachable::LOCATOR_NAME++ GlobalID.parse(uri)+ rescue ArgumentError, JSON::ParserErrornilend- def belongs_to_text?(rich_text, record)- if record.is_a?(ActiveStorage::Blob)- ActiveStorage::Attachment.exists?(record: rich_text, name: 'embeds', blob: record)- else- carrier_organisation_id = rich_text.record.try(:organisation_id)- carrier_organisation_id.present? && record.try(:organisation_id) == carrier_organisation_id+ def embeds_of(rich_text) = rich_text.embeds_attachments.map(&:blob)++ # An image node whose payload is not readable: the one embed with its filename and+ # filesize that no other node of the text has taken. Two candidates are no answer.+ def embed_by_attributes(rich_text, node, claimed)+ return if node['filename'].blank? || node['filesize'].blank?++ matches = (embeds_of(rich_text) - claimed).select do |blob|+ blob.filename.to_s == node['filename'] && blob.byte_size.to_s == node['filesize']end+ matches.one? ? matches.first : nil+ end++ def referenced_target(carrier, gid)+ return unless carrier.respond_to?(:outgoing_references)++ carrier.outgoing_references.find { |ref| ref.target_type == gid.model_name && ref.target_id == gid.model_id }&.targetend- # Model and id, never gid.to_s: on a SignedGlobalID that signs the id again.- def list(kind, rich_text, gid)- @result[kind] << { rich_text_id: rich_text.id, record_type: rich_text.record_type, record_id: rich_text.record_id,- target_type: gid.model_name, target_id: gid.model_id }+ def unmatched(rich_text, node, gid, reason)+ @result.unmatched << { rich_text_id: rich_text.id, record_type: rich_text.record_type, record_id: rich_text.record_id,+ reason: reason, target_type: gid&.model_name, target_id: gid&.model_id,+ filename: node['filename'] }nilenddiff --git a/lib/tasks/rich_text_embeds.rake b/lib/tasks/rich_text_embeds.rakeindex ac3d000..7ac1c51 100644--- a/lib/tasks/rich_text_embeds.rake+++ b/lib/tasks/rich_text_embeds.rake@@ -12,25 +12,20 @@ namespace :rich_text_embeds doputs " #{result.dry_run ? 'Würden geändert' : 'Geändert'}: #{result.changed}"end- # After a secret_key_base rotation (RichTextSgidResign). The old key comes from the- # environment of this one run only; read it with `read -rs OLD_SECRET_KEY_BASE` and- # export it, so it lands neither in the shell history nor in a file.- desc 'Attachment-Knoten mit dem aktuellen Schlüssel neu signieren. OLD_SECRET_KEY_BASE setzen, zuerst DRY_RUN=1.'+ # After a secret_key_base rotation (RichTextSgidResign): re-signs the attachment nodes+ # whose target the app stored next to the text. Needs no old key.+ desc 'Attachment-Knoten nach einem Tausch von secret_key_base neu signieren. Immer zuerst mit DRY_RUN=1.'task resign_sgids: :environment do- old_secret_key_base = ENV.fetch('OLD_SECRET_KEY_BASE', '')- abort 'OLD_SECRET_KEY_BASE fehlt (der Schlüssel VOR dem Tausch).' if old_secret_key_base.blank?- abort 'OLD_SECRET_KEY_BASE ist der aktuelle Schlüssel.' if old_secret_key_base == Rails.application.secret_key_base-- result = RichTextSgidResign.call(old_secret_key_base: old_secret_key_base, dry_run: ENV['DRY_RUN'] == '1')+ result = RichTextSgidResign.call(dry_run: ENV['DRY_RUN'] == '1')puts format('rich_text_embeds:resign_sgids%<mode>s', mode: result.dry_run ? ' (DRY_RUN, nichts geschrieben)' : '')puts " Rich-Texte mit Attachment-Knoten: #{result.scanned}"puts " #{result.dry_run ? 'Würden geändert' : 'Geändert'}: #{result.changed} Texte, #{result.resigned} Knoten"- puts " Mit keinem der beiden Schlüssel lesbar: #{result.unreadable}"- { gone: 'Ziel fehlt (Blob gelöscht, aus der Sicherung holen)', refused: 'Abgelehnt (gehört nicht zu diesem Text)' }.each do |kind, label|- rows = result[kind]- puts " #{label}: #{rows.size}"+ puts " Nicht zugeordnet: #{result.unmatched.size}"+ result.unmatched.group_by { |row| row[:reason] }.each do |reason, rows|+ puts " #{reason}: #{rows.size}"rows.each do |row|- puts " #{row[:record_type]} #{row[:record_id]} (Rich-Text #{row[:rich_text_id]}) -> #{row[:target_type]} #{row[:target_id]}"+ target = row[:target_id] ? "#{row[:target_type]} #{row[:target_id]}" : "Datei #{row[:filename].inspect}"+ puts " #{row[:record_type]} #{row[:record_id]} (Rich-Text #{row[:rich_text_id]}) -> #{target}"endendenddiff --git a/spec/services/rich_text_sgid_resign_spec.rb b/spec/services/rich_text_sgid_resign_spec.rbindex 849246d..f601e85 100644--- a/spec/services/rich_text_sgid_resign_spec.rb+++ b/spec/services/rich_text_sgid_resign_spec.rb@@ -3,17 +3,16 @@require 'rails_helper'# What: RichTextSgidResign re-signs the attachment nodes of stored rich texts whose sgid-# was signed with the previous secret_key_base.+# was signed with a previous secret_key_base, without that key.# Type: Service spec (2026-10-20).# Why: A secret_key_base rotation turns every stored image and wiki mention into a-# "☒". The old key comes only from the one run of the task, so a node is accepted-# only when the old key signed it AND its target belongs to this text: an image-# only when it is attached to THIS text, a mention only within the text's own-# organisation. Others may hold a rotated key; a sgid forged with it must gain nothing.+# "☒". A stale signature proves nothing, so a node is re-signed only when what the+# app stored next to the text names the same target: an embed of this text for an+# image, an outgoing reference of the note for a mention. A forged node, or one+# that names anything else, must gain nothing. A Marshal payload is never loaded.RSpec.describe RichTextSgidResign, :ci_guard dolet(:organisation) { create(:organisation) }let(:project) { create(:project, organisation: organisation) }- let(:old_secret_key_base) { Rails.application.secret_key_base }before { ActsAsTenant.current_tenant = organisation }@@ -38,14 +37,34 @@ RSpec.describe RichTextSgidResign, :ci_guard dodef mention(note) = node(note, ' content-type="application/vnd.actiontext.mention"')- # What a rotation does: the sgids written so far were signed with the key- # the application booted with, every one minted from now on uses the new key.+ # What a rotation does: the sgids written so far were signed with the key the+ # application booted with, every one minted from now on uses the new key.def rotate_secret_key_base!- key = ActiveSupport::KeyGenerator.new('a-rotated-secret-key-base' * 4, iterations: 1000)- .generate_key('signed_global_ids')- SignedGlobalID.verifier = GlobalID::Verifier.new(key)+ SignedGlobalID.verifier = GlobalID::Verifier.new(key_for('a-rotated-secret-key-base'))end+ def key_for(secret) = ActiveSupport::KeyGenerator.new(secret * 4, iterations: 1000).generate_key('signed_global_ids')++ # The self-describing metadata of older globalid releases, serialised with Marshal.+ def marshal_sgid(record)+ GlobalID::Verifier.new(key_for('an-old-secret-key-base'), serializer: Marshal)+ .generate({ 'gid' => record.to_gid.to_s, 'purpose' => 'attachable', 'expires_at' => nil })+ end++ # A payload anybody can write: the current envelope with a made-up digest.+ def unsigned_sgid(record)+ payload = { _rails: { data: record.to_gid.to_s, pur: 'attachable' } }.to_json+ "#{Base64.urlsafe_encode64(payload)}--#{'0' * 40}"+ end++ def ticket_with_body(html)+ create(:ticket, project: project, organisation: organisation, content: '<div>x</div>').tap do |ticket|+ ticket.content.update_column(:body, html)+ end+ end++ def attach_embed(rich_text, blob) = ActiveStorage::Attachment.create!(record: rich_text, name: 'embeds', blob: blob)+def sgids_of(record)Nokogiri::HTML.fragment(record.reload.content.body.to_html).css('action-text-attachment').map { |n| n['sgid'] }end@@ -56,7 +75,7 @@ RSpec.describe RichTextSgidResign, :ci_guard dofalseend- def run(dry_run: false) = described_class.call(old_secret_key_base: old_secret_key_base, dry_run: dry_run)+ def run(dry_run: false) = described_class.call(dry_run: dry_run)it 'brings back an image of the text and renders it instead of the glyph' doblob = png_blob@@ -64,9 +83,7 @@ RSpec.describe RichTextSgidResign, :ci_guard dorotate_secret_key_base!expect(Ticket.find(ticket.id).content.to_s).to include('☒')- result = run-- expect(result.resigned).to eq(1)+ expect(run.resigned).to eq(1)expect(sgids_of(ticket)).to all(satisfy { |sgid| resolves?(sgid) })expect(Ticket.find(ticket.id).content.to_s).not_to include('☒')end@@ -76,9 +93,7 @@ RSpec.describe RichTextSgidResign, :ci_guard dostale = sgids_of(ticket)rotate_secret_key_base!- result = run(dry_run: true)-- expect(result.to_h).to include(changed: 1, resigned: 1, dry_run: true)+ expect(run(dry_run: true).to_h).to include(changed: 1, resigned: 1, dry_run: true)expect(sgids_of(ticket)).to eq(stale)end@@ -90,7 +105,7 @@ RSpec.describe RichTextSgidResign, :ci_guard doexpect(run.to_h).to include(changed: 0, resigned: 0)end- it 'brings back a wiki mention within the organisation and re-derives the markdown body' do+ it 'brings back a wiki mention the note references and re-derives the markdown body' dotarget = create(:note, project: project, organisation: organisation, title: 'Zielnotiz')note = create(:note, project: project, organisation: organisation, content: "<div>Siehe</div>#{mention(target)}")rotate_secret_key_base!@@ -101,45 +116,61 @@ RSpec.describe RichTextSgidResign, :ci_guard doexpect(note.reload.markdown_body).to eq(Wiki::MarkdownSerializer.new.to_markdown(note.content.body.to_html))end- it 'refuses an image that is not attached to this text' do- foreign = png_blob('fremd.png')- ticket = create(:ticket, project: project, organisation: organisation, content: '<div>x</div>')- ticket.content.update_column(:body, "<div>x</div>#{node(foreign)}")+ it 'refuses a mention the note does not reference, even within the organisation' do+ target = create(:note, project: project, organisation: organisation, title: 'Nie verlinkt')+ note = create(:note, project: project, organisation: organisation, content: '<div>x</div>')+ note.content.update_column(:body, "<div>x</div>#{mention(target)}")rotate_secret_key_base!result = runexpect(result.resigned).to eq(0)- expect(result.refused).to contain_exactly(hash_including(rich_text_id: ticket.content.id, target_type: 'ActiveStorage::Blob', target_id: foreign.id))- expect(resolves?(sgids_of(ticket).first)).to be(false)+ expect(result.unmatched).to contain_exactly(hash_including(reason: :no_reference, target_type: 'Note', target_id: target.id))end- it 'refuses a mention of a record in another organisation' do- other = create(:organisation)- target = ActsAsTenant.with_tenant(other) do- create(:note, project: create(:project, organisation: other), organisation: other, title: 'Fremd')- end- note = create(:note, project: project, organisation: organisation, content: '<div>x</div>')- note.content.update_column(:body, "<div>x</div>#{mention(target)}")+ it 'refuses an image that is not attached to this text' do+ foreign = png_blob('fremd.png')+ ticket = ticket_with_body("<div>x</div>#{node(foreign)}")rotate_secret_key_base!result = runexpect(result.resigned).to eq(0)- expect(result.refused).to contain_exactly(hash_including(target_type: 'Note', target_id: target.id))+ expect(result.unmatched).to contain_exactly(+ hash_including(rich_text_id: ticket.content.id, reason: :not_attached, target_type: 'ActiveStorage::Blob', target_id: foreign.id)+ )+ expect(resolves?(sgids_of(ticket).first)).to be(false)end- it 'refuses a sgid the old key did not sign, without decoding it' do- blob = png_blob- ticket = create(:ticket, project: project, organisation: organisation, content: "<div>x</div>#{node(blob)}")- rotate_secret_key_base!-- result = described_class.call(old_secret_key_base: 'not-the-old-key' * 8, dry_run: false)+ it 'gives a forged, unsigned node nothing it could not already show' do+ foreign = png_blob('fremd.png')+ ticket = ticket_with_body(%(<div>x</div><action-text-attachment sgid="#{unsigned_sgid(foreign)}"></action-text-attachment>))- expect(result.to_h).to include(resigned: 0, unreadable: 1)+ expect(run.resigned).to eq(0)expect(resolves?(sgids_of(ticket).first)).to be(false)end+ it 'matches a Marshal node to its embed by filename and filesize, without loading the payload' do+ blob = png_blob('alt.png')+ ticket = ticket_with_body(%(<div>x</div><action-text-attachment sgid="#{marshal_sgid(blob)}" filename="alt.png" filesize="#{blob.byte_size}"></action-text-attachment>))+ attach_embed(ticket.content, blob)+ expect(Marshal).not_to receive(:load)++ expect(run.resigned).to eq(1)+ expect(ActionText::Attachable.from_attachable_sgid(sgids_of(ticket).first)).to eq(blob)+ end++ it 'lists a Marshal node it cannot match' do+ blob = png_blob('alt.png')+ ticket = ticket_with_body(%(<div>x</div><action-text-attachment sgid="#{marshal_sgid(blob)}" filename="anders.png" filesize="1"></action-text-attachment>))+ attach_embed(ticket.content, blob)++ result = run++ expect(result.resigned).to eq(0)+ expect(result.unmatched).to contain_exactly(hash_including(reason: :unreadable, filename: 'anders.png', target_id: nil))+ end+it 'lists an image whose blob is gone so it can be restored from the backup' doblob = png_blobticket = create(:ticket, project: project, organisation: organisation, content: "<div>x</div>#{node(blob)}")@@ -150,10 +181,8 @@ RSpec.describe RichTextSgidResign, :ci_guard doresult = runexpect(result.resigned).to eq(0)- expect(result.gone).to contain_exactly(hash_including(rich_text_id: ticket.content.id, target_type: 'ActiveStorage::Blob', target_id: blob.id))- end-- it 'refuses to run without the old key' do- expect { described_class.call(old_secret_key_base: '', dry_run: true) }.to raise_error(ArgumentError)+ expect(result.unmatched).to contain_exactly(+ hash_including(rich_text_id: ticket.content.id, reason: :not_attached, target_type: 'ActiveStorage::Blob', target_id: blob.id)+ )endend