Skip to content

Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.

fix: review round 1 of #121, take a mark only where its match shows at it

Guido Boyke0c56a6c2b2dc
diff --git a/app/controllers/organisations/meeting_term_corrections_controller.rb b/app/controllers/organisations/meeting_term_corrections_controller.rb
index 043c2af..2382497 100644
--- a/app/controllers/organisations/meeting_term_corrections_controller.rb
+++ b/app/controllers/organisations/meeting_term_corrections_controller.rb
@@ -102,7 +102,7 @@ module Organisations
def build_dialog(heard:)
Meetings::TermCorrectionDialog.new(
transcript: @meeting.transcript, participants: participants, mark: mark_param, heard: heard,
- displayed_html: helpers.meeting_transcript_html(@meeting, participants), digest: scalar_param(:digest)
+ render: ->(text) { helpers.meeting_transcript_html(text, participants) }, digest: scalar_param(:digest)
)
end
diff --git a/app/helpers/meetings_helper.rb b/app/helpers/meetings_helper.rb
index b18c1aa..03a580a 100644
--- a/app/helpers/meetings_helper.rb
+++ b/app/helpers/meetings_helper.rb
@@ -268,15 +268,22 @@ module MeetingsHelper
# participant list. The caller passes the participants it ALREADY loaded for the
# header chips — never re-query here, that would be one extra query per render.
def meeting_transcript_source(meeting, participants)
- meeting_markdown_source(meeting.transcript, speakers: participants.map(&:display_name))
+ meeting_transcript_markdown(meeting.transcript, participants)
end
- # The HTML the transcript block of the page holds: the component's own render
- # of meeting_transcript_source. Meetings::TranscriptMark reads the text the
- # browser shows out of it, to find a mark of the reader in the stored
- # transcript (K4b); rendered the same way, the two texts are the same.
- def meeting_transcript_html(meeting, participants)
- Neo::MarkdownPreviewComponent.new(markdown: meeting_transcript_source(meeting, participants)).rendered_markdown
+ # The Markdown the page shows for a transcript text: the one place that
+ # prepares it, for the page and for the correction of a term alike.
+ def meeting_transcript_markdown(transcript, participants)
+ meeting_markdown_source(transcript, speakers: participants.map(&:display_name))
+ end
+
+ # The HTML the transcript block of the page holds for a transcript text: the
+ # component's own render of meeting_transcript_markdown.
+ # Meetings::TranscriptMark reads the text the browser shows out of it, and
+ # puts a probe of the stored text through it, to find a mark of the reader
+ # (K4b); rendered the same way, the texts are the same.
+ def meeting_transcript_html(transcript, participants)
+ Neo::MarkdownPreviewComponent.new(markdown: meeting_transcript_markdown(transcript, participants)).rendered_markdown
end
# Plain-text source for the header "Abriss": the AI summary if present, else the
diff --git a/app/services/CLAUDE.md b/app/services/CLAUDE.md
index 57c7a7e..a282601 100644
--- a/app/services/CLAUDE.md
+++ b/app/services/CLAUDE.md
@@ -198,9 +198,12 @@ Verzeichnis-Übersicht (Auswahl):
und Ersetzen einer gehörten Form (ganze Wörter nach Unicode, NICHT `\b`; Fall wie markiert;
mehrere Wörter nur innerhalb der Zeile; nie in der Sprecherbezeichnung, deren Regel
`speaker_markup.rb#label` mit dem Fettsetzen teilt; nie im Frontmatter). `#occurrences` speist
- Dialog und Ersetzung zugleich. `transcript_mark.rb` findet eine Markierung des Browsers im
- GESPEICHERTEN Transkript: Text der Seite aus demselben HTML (`MeetingsHelper#meeting_transcript_html`),
- n-ter Treffer dort = n-ter Treffer im Speicher, sonst Ablehnung. `term_correction_dialog.rb`
+ Dialog und Ersetzung zugleich; Unsichtbares (`GlossaryText::INVISIBLE`) verbindet wie ein Buchstabe.
+ `transcript_mark.rb` findet eine Markierung des Browsers im GESPEICHERTEN Transkript: Text der
+ Seite aus derselben Darstellung (`MeetingsHelper#meeting_transcript_html`), n-ter Treffer dort =
+ n-ter Treffer im Speicher. ⚠️ Gleiche Zahl ist kein Beweis (Link-Ziel versteckt, Entität erzeugt):
+ der Treffer wird mit einer Marke an seiner Stelle neu dargestellt und nur genommen, wenn die Marke
+ genau an der Markierung steht (Review-Runde 1 von #121), sonst Ablehnung. `term_correction_dialog.rb`
ist, was der Dialog zeigt (Stelle, Zahl, Sprecher, Digest), geteilt von `new`, `count`, `create`.
* `people/` — `directory.rb`, das Lese-Objekt der Personenliste (seit 26.10.2026): Mitglieder
über `MembershipPolicy::Scope` (dieselbe Reichweite wie die Mitgliederseite), dazu die
diff --git a/app/services/meetings/term_correction_dialog.rb b/app/services/meetings/term_correction_dialog.rb
index 4a3a3dc..129237d 100644
--- a/app/services/meetings/term_correction_dialog.rb
+++ b/app/services/meetings/term_correction_dialog.rb
@@ -22,9 +22,10 @@ module Meetings
# mark: { text:, before:, after:, offset: } as the browser sent it.
# heard: the heard form as typed, nil for the first opening (the mark itself).
# digest: the digest the browser brought (the page's, or the dialog's last count).
- def initialize(transcript:, displayed_html:, participants:, mark:, heard:, digest:)
+ # render: the page's own rendering of a transcript text (see Meetings::TranscriptMark).
+ def initialize(transcript:, render:, participants:, mark:, heard:, digest:)
@transcript = transcript.to_s
- @displayed_html = displayed_html
+ @render = render
@participants = participants
@mark = mark
@heard = heard.nil? ? mark[:text].to_s.strip : heard.to_s
@@ -77,7 +78,7 @@ module Meetings
private
def mark_result
- @mark_result ||= TranscriptMark.resolve(transcript: @transcript, displayed_html: @displayed_html,
+ @mark_result ||= TranscriptMark.resolve(transcript: @transcript, render: @render,
participants: @participants, **@mark.slice(:text, :before, :after, :offset))
end
diff --git a/app/services/meetings/term_replacement.rb b/app/services/meetings/term_replacement.rb
index db16349..47aee29 100644
--- a/app/services/meetings/term_replacement.rb
+++ b/app/services/meetings/term_replacement.rb
@@ -27,6 +27,12 @@ module Meetings
class TermReplacement
# A letter (with its combining marks) or a digit of any script.
WORD_CHARACTER = '[\p{L}\p{M}\p{N}]'
+ # What shows nothing (a zero-width space, a soft hyphen, a bidi mark: the
+ # set GlossaryText drops) joins what stands around it, as the reader sees
+ # it: "Lalabeis<zero-width space>Konto" reads as one word. The boundary
+ # looks past it, before the form up to two such characters (a look-behind
+ # has a fixed width), after it any number.
+ INVISIBLE = GlossaryText::INVISIBLE.source
# Between the words of a form: horizontal whitespace only.
WORD_GAP = '[[:blank:]]+'
@@ -142,7 +148,8 @@ module Meetings
def pattern
body = @words.map { |word| Regexp.escape(word) }.join(WORD_GAP)
- Regexp.new("(?<!#{WORD_CHARACTER})#{body}(?!#{WORD_CHARACTER})")
+ before = "(?<!#{WORD_CHARACTER})(?<!#{WORD_CHARACTER}#{INVISIBLE})(?<!#{WORD_CHARACTER}#{INVISIBLE}#{INVISIBLE})"
+ Regexp.new("#{before}#{body}(?!#{INVISIBLE}*#{WORD_CHARACTER})")
end
# The frontmatter's lines are no content: nobody sees or marks them. A
diff --git a/app/services/meetings/transcript_mark.rb b/app/services/meetings/transcript_mark.rb
index 66e7dfb..4452804 100644
--- a/app/services/meetings/transcript_mark.rb
+++ b/app/services/meetings/transcript_mark.rb
@@ -10,17 +10,25 @@
# So the browser sends what the reader marked, up to CONTEXT_LENGTH
# characters of the text the page shows before and after it, and how many
# characters of that text stand before it. This class reads the same text out
-# of the same HTML (MeetingsHelper#meeting_transcript_html), finds the mark
-# there (the offset first, the context as its proof; else the one place with
-# that context), asks which whole-word match of the form it is, and takes the
-# match with the same number in the stored transcript
+# of the same rendering (MeetingsHelper#meeting_transcript_html), finds the
+# mark there (the offset first, the context as its proof; else the one place
+# with that context), asks which whole-word match of the form it is, and
+# takes the match with the same number in the stored transcript
# (Meetings::TermReplacement#scan, the rule of counting and replacing).
#
+# The same number is no proof. Markdown can hide one match (a link target, a
+# reference definition, a tag the sanitizer drops) and show another (an
+# entity, a link text right at the word): the counts agree, the order does
+# not (review round 1 of #121). So the match found is put back through the
+# page's own rendering with a marker in its place, and it is taken only when
+# the page then shows the marker exactly where the reader marked, with all
+# the rest of its text unchanged.
+#
# It refuses rather than guesses:
# * not_found: no place shows the mark with its context, or several do;
# * not_a_word: the mark is part of a longer word;
# * ambiguous: the page and the stored text hold the form a different number
-# of times (Markdown hid some of it, a link target or an escaped sign);
+# of times, or the match with the same number does not show at the mark;
# * speaker_label: the mark stands in the speaker label of its line;
# * line_break, too_long: no heard form at all.
#
@@ -45,12 +53,13 @@ module Meetings
new(**).resolve
end
- # transcript: as stored. displayed_html: what the page shows of it.
- # participants: the meeting's (anything with a display_name). text,
- # before, after: strings from the browser. offset: an Integer or nil.
- def initialize(transcript:, displayed_html:, participants:, text:, before:, after:, offset:)
+ # transcript: as stored. render: what the page shows for a transcript
+ # text, as HTML (a callable; the caller hands in the page's own
+ # rendering). participants: the meeting's (anything with a display_name).
+ # text, before, after: strings from the browser. offset: an Integer or nil.
+ def initialize(transcript:, render:, participants:, text:, before:, after:, offset:)
@transcript = transcript.to_s
- @displayed_html = displayed_html.to_s
+ @render = render
@participants = participants
raw = text.to_s
lead = raw[/\A[[:space:]]*/]
@@ -80,20 +89,35 @@ module Meetings
ordinal = shown.index { |match| match.line_index == line_index && match.byte_offset == column }
return refuse(:not_a_word) unless ordinal
- stored = TermReplacement.new(text: @transcript, heard: @mark, participants: @participants).scan
- return refuse(:ambiguous) unless stored.size == shown.size
+ stored = TermReplacement.new(text: @transcript, heard: @mark, participants: @participants)
+ return refuse(:ambiguous) unless stored.scan.size == shown.size
- match = stored[ordinal]
+ match = stored.scan[ordinal]
return refuse(:speaker_label) if match.in_label
+ return refuse(:ambiguous) unless shows_at_mark?(stored, match, start)
Result.new(place: Place.new(line_index: match.line_index, char_offset: match.char_offset, length: match.length),
error: nil)
end
+ # The proof of the number: the stored match, replaced by a marker and
+ # shown through the page's own rendering, shows the marker exactly at the
+ # mark, and every other character of the page as before. The marker is
+ # letters and digits only, so Markdown reads it as the word it replaces.
+ def shows_at_mark?(stored, match, start)
+ marker = "Q#{SecureRandom.alphanumeric(16)}Q"
+ shown = text_of(@render.call(stored.replace(with: marker, only: match).text))
+ shown == visible.byteslice(0, start) + marker + visible.byteslice(start + @mark.bytesize, visible.bytesize)
+ end
+
# The text the content element of the page holds, as the browser reads it
# (the text of every node, in order).
def visible
- @visible ||= Nokogiri::HTML5.fragment(@displayed_html).text
+ @visible ||= text_of(@render.call(@transcript))
+ end
+
+ def text_of(html)
+ Nokogiri::HTML5.fragment(html.to_s).text
end
# Where the mark starts in the text the page shows, in bytes, or nil.
diff --git a/spec/requests/organisations/meeting_term_corrections_spec.rb b/spec/requests/organisations/meeting_term_corrections_spec.rb
index 020059d..93a551b 100644
--- a/spec/requests/organisations/meeting_term_corrections_spec.rb
+++ b/spec/requests/organisations/meeting_term_corrections_spec.rb
@@ -53,7 +53,7 @@ RSpec.describe 'Organisations::MeetingTermCorrections', :ci_guard, type: :reques
# What the browser sends for the nth place of `text` in the text the page shows.
def browser_mark(text = 'Lalabeis', nth: 1)
- html = ApplicationController.helpers.meeting_transcript_html(meeting.reload, meeting.meeting_participants.in_row_order.to_a)
+ html = ApplicationController.helpers.meeting_transcript_html(meeting.reload.transcript, meeting.meeting_participants.in_row_order.to_a)
shown = Nokogiri::HTML5.fragment(html).text
start = shown.to_enum(:scan, text).map { Regexp.last_match.begin(0) }.fetch(nth)
{ text: text, before: shown[[start - 40, 0].max...start], after: shown[start + text.length, 40], offset: start }
@@ -340,6 +340,22 @@ RSpec.describe 'Organisations::MeetingTermCorrections', :ci_guard, type: :reques
end
end
+ # The area gate of ProjectController (every action here is a write) stands
+ # in front of Pundit: a reader stays out even where the policy would let
+ # them in (review round 1 of #121: nothing tested the gate on its own).
+ it 'keeps a reader out at the area gate, before the policy is asked' do
+ sign_in writer('manage_glossary', access: :read_only)
+ allow_any_instance_of(MeetingPolicy).to receive(:update?).and_return(true)
+
+ get new_path, headers: frame
+ expect(response).to have_http_status(:forbidden)
+ get count_path
+ expect(response).to have_http_status(:forbidden)
+ post_correction
+ expect(response).to have_http_status(:forbidden)
+ expect(meeting.reload.transcript).to eq(transcript)
+ end
+
it 'answers 403 in an archived project to a writer' do
user = writer('manage_glossary')
project.update_columns(archived_at: Time.current)
diff --git a/spec/services/meetings/term_replacement_spec.rb b/spec/services/meetings/term_replacement_spec.rb
index 155922b..fe519e7 100644
--- a/spec/services/meetings/term_replacement_spec.rb
+++ b/spec/services/meetings/term_replacement_spec.rb
@@ -64,6 +64,15 @@ RSpec.describe Meetings::TermReplacement do
expect(count('Läuft das auf Version 2.0?', '2.0?')).to eq(1)
end
+ # Review round 1 of #121: what shows nothing (GlossaryText::INVISIBLE: a
+ # zero-width space, a soft hyphen, a bidi mark) joins what stands around it,
+ # as the reader sees it and as the glossary compares it.
+ it 'reads characters that show nothing as part of the word around them' do
+ expect(count("Das Lalabeis\u200BKonto", 'Lalabeis')).to eq(0)
+ expect(count("Lala\u00ADbeis", 'Lala')).to eq(0)
+ expect(count("Lalabeis\u200E und \u200BLalabeis", 'Lalabeis')).to eq(2)
+ end
+
it 'counts digits of every script as part of a word' do
expect(count('Raum 12 und Raum 123', 'Raum 12')).to eq(1)
expect(count("Lalabeis\u0663 und Lalabeis", 'Lalabeis')).to eq(1)
diff --git a/spec/services/meetings/transcript_mark_spec.rb b/spec/services/meetings/transcript_mark_spec.rb
index 1631280..dd1c7dc 100644
--- a/spec/services/meetings/transcript_mark_spec.rb
+++ b/spec/services/meetings/transcript_mark_spec.rb
@@ -15,13 +15,14 @@ RSpec.describe Meetings::TranscriptMark do
let(:participants) { [participant('Anna Ananas'), participant('Ben Birne')] }
- def displayed_html(transcript)
- ApplicationController.helpers.meeting_transcript_html(Struct.new(:transcript).new(transcript), participants)
+ # The page's own rendering of a transcript text, as the controller hands it in.
+ def render
+ ->(text) { ApplicationController.helpers.meeting_transcript_html(text, participants) }
end
# The text the browser shows: what the content element of the page holds.
def visible(transcript)
- Nokogiri::HTML5.fragment(displayed_html(transcript)).text
+ Nokogiri::HTML5.fragment(render.call(transcript)).text
end
# What the browser sends for the nth (0-based) place of `mark` in the text
@@ -33,7 +34,7 @@ RSpec.describe Meetings::TranscriptMark do
end
def resolve(transcript, **mark)
- described_class.resolve(transcript: transcript, displayed_html: displayed_html(transcript), participants: participants, **mark)
+ described_class.resolve(transcript: transcript, render: render, participants: participants, **mark)
end
let(:transcript) do
@@ -101,6 +102,48 @@ RSpec.describe Meetings::TranscriptMark do
expect(resolve(text, **browser_mark(text, 'lalabeis')).error).to eq(:ambiguous)
end
+ # Review round 1 of #121: Markdown can hide one match (a link target, a
+ # reference definition, a tag the sanitizer drops) and show another (an
+ # entity, a link text right at the word). The counts agree, the order does
+ # not, and the n-th match was another word. The match is put back through the
+ # page's own rendering with a marker in its place; only where the marker
+ # shows at the mark with everything around it unchanged is it taken.
+ describe 'Markdown that hides one match and shows another' do
+ it 'refuses a mark the stored text holds elsewhere (entity shown, reference target hidden)' do
+ text = "Anna: &#76;alabeis ist neu.\nBen: Lalabeis ist gut.\n\n[x]: https://example.com/Lalabeis"
+
+ expect(resolve(text, **browser_mark(text, 'Lalabeis', nth: 0)).error).to eq(:ambiguous)
+ expect(resolve(text, **browser_mark(text, 'Lalabeis', nth: 1)).error).to eq(:ambiguous)
+ end
+
+ it 'refuses a mark that would land in a link target (target hidden, link text at the word shown)' do
+ text = "Anna: Siehe [Wiki](https://example.com/Lalabeis) und [Lala](https://example.com)beis heute.\nBen: Lalabeis ist gut."
+
+ expect(resolve(text, **browser_mark(text, 'Lalabeis', nth: 0)).error).to eq(:ambiguous)
+ end
+
+ it 'refuses a mark that would land in a tag the page drops' do
+ text = "Anna: <Lalabeis> und &#76;alabeis.\nBen: ok"
+
+ expect(resolve(text, **browser_mark(text, 'Lalabeis', nth: 0)).error).to eq(:ambiguous)
+ end
+
+ it 'still finds a word in a link text, a bold word and a word after an entity' do
+ text = "Ben: Siehe [Lalabeis](https://example.com/wiki) heute.\nAnna: **Lalabeis** &amp; Lalabeis."
+
+ places = (0..2).map { |nth| resolve(text, **browser_mark(text, 'Lalabeis', nth: nth)).place&.to_h&.values_at(:line_index, :char_offset) }
+ expect(places).to eq([[0, 12], [1, 8], [1, 25]])
+ end
+ end
+
+ it 'takes the context after the mark into account as well as the one before it' do
+ text = "Ben: ja Lalabeis heute\nBen: ja Lalabeis morgen\n"
+
+ result = resolve(text, text: 'Lalabeis', before: 'ja ', after: ' heute', offset: 0)
+
+ expect(result.place.to_h.values_at(:line_index, :char_offset)).to eq([0, 8])
+ end
+
it 'refuses a mark over a line break and a mark too long for the glossary' do
two_lines = "Anna: eins Lala\nBeis zwei\n"
long = "Anna: #{'x' * 300}\n"