Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.
Ruby agplai-assistantdockerdsgvogdprhotwireissue-trackerjira-alternativeknowledge-basellmmcp-servermeeting-notesopen-sourceproject-managementredmine-alternativeruby-on-railsself-hostedticketingtime-tracking
Fix review round 2 of #49: keep long analyses working and tighten the truncation contract
Guido Boykecb03a6b86d41
diff --git a/app/services/lalabase/CLAUDE.md b/app/services/lalabase/CLAUDE.mdindex 8f0b3e2..630b33b 100644--- a/app/services/lalabase/CLAUDE.md+++ b/app/services/lalabase/CLAUDE.md@@ -235,7 +235,7 @@ Wichtig:(etwa `reasoning_effort`) schickt nur ein Seam mit Registry-Pin (`:customer_description`).Endet eine Antwort am Limit (`stop_reason 'length'`), ist sie abgeschnitten:`Meetings::BaseLlmService` meldet dann den Fehler `truncated` statt den Teiltext zu- speichern. Kam vor dem Limit gar kein Text, werfen Vertex und Anthropic+ speichern. Kam vor dem Limit gar kein Text, werfen OpenAI, Vertex und Anthropic`Provider::TruncatedError` (eine Unterklasse von `Provider::Error`, mit dem Ergebnis samtVerbrauch); `BaseLlmService` bucht ihn und meldet ebenfalls `truncated`. ⚠️ Der Assistent(`Tools::Runner`) und die Teilnehmer-Erkennung werten `'length'` noch nicht aus.diff --git a/app/services/lalabase/llm/open_ai_provider.rb b/app/services/lalabase/llm/open_ai_provider.rbindex 4ba4ec0..0416b88 100644--- a/app/services/lalabase/llm/open_ai_provider.rb+++ b/app/services/lalabase/llm/open_ai_provider.rb@@ -95,15 +95,22 @@ module Lalabasecontent = msg['content']raw_tool_calls = Array(msg['tool_calls'])- raise Error, "OpenAI returned empty response: #{response.inspect}" if content.nil? && raw_tool_calls.empty?-- {+ result = {content: content.to_s.strip,tool_calls: raw_tool_calls.empty? ? nil : raw_tool_calls.map { |tc| parse_tool_call(tc) },stop_reason: normalise_stop_reason(response.dig('choices', 0, 'finish_reason')),model_used: model,token_usage: extract_token_usage(response, model: model)}+ if content.nil? && raw_tool_calls.empty?+ message = "OpenAI returned empty response: #{response.inspect}"+ # Stopped at the limit before any text: see Provider::TruncatedError.+ raise TruncatedError.new(message, result: result) if result[:stop_reason] == 'length'++ raise Error, message+ end++ resultrescue Faraday::UnauthorizedError, Faraday::TooManyRequestsError => eraise mapped_error(e)enddiff --git a/app/services/lalabase/llm/provider.rb b/app/services/lalabase/llm/provider.rbindex ce00613..9541182 100644--- a/app/services/lalabase/llm/provider.rb+++ b/app/services/lalabase/llm/provider.rb@@ -42,7 +42,7 @@ module Lalabasedef initialize(message = nil, result: {})super(message)- @result = result+ @result = result || {}endend@@ -126,7 +126,8 @@ module Lalabase# thinks first puts its own reserve on top (GoogleVertexProvider::THINKING_BUDGET,# OpenAiProvider::REASONING_RESERVE); callers never add one themselves. An# answer that hit the limit before any text raises TruncatedError (with the- # result and its usage) where a provider would otherwise raise "empty content".+ # result and its usage) in #chat of the OpenAI, Anthropic and Vertex+ # providers, where they would otherwise raise "empty content".# @param temperature [Float]# @return [Hash] {content:, tool_calls: nil | Array, stop_reason: String,# model_used:, token_usage: {...}}.diff --git a/app/services/meetings/analysis_service.rb b/app/services/meetings/analysis_service.rbindex 3648ff7..4113482 100644--- a/app/services/meetings/analysis_service.rb+++ b/app/services/meetings/analysis_service.rb@@ -16,9 +16,12 @@ module Meetings# A literal 'gpt-4o-mini' would be a runtime 404 on a Mistral/IONOS endpoint,# and the task profile is provider-neutral anyway: a prompt in, German prose# out — every chat model can do that.- # An analysis (protocol, decision list, redacted version) can run longer than a- # 150-word summary; still bounded, and the SOURCE_CHAR_LIMIT cap bounds input.- MAX_TOKENS = 1_500+ # An analysis (protocol, decision list, redacted version) can run much longer+ # than a 150-word summary: a protocol of a long meeting does not fit into+ # 1,500 tokens. Since an answer cut at the limit is an error (BaseLlmService),+ # a small budget would fail every long analysis on a model without a thinking+ # reserve. Still bounded; SOURCE_CHAR_LIMIT bounds the input.+ MAX_TOKENS = 8_000TEMPERATURE = 0.5def self.call(analysis:)diff --git a/config/locales/services.de.yml b/config/locales/services.de.ymlindex b1f88e3..1b0f7d6 100644--- a/config/locales/services.de.yml+++ b/config/locales/services.de.yml@@ -131,7 +131,7 @@ de:empty_result: "Die KI hat kein Ergebnis zurückgegeben. Bitte versuchen Sie es erneut."generic_error: "Die Analyse konnte nicht erstellt werden. Bitte versuchen Sie es später erneut."quota_exhausted: "Das KI-Kontingent ist aufgebraucht. Die Analyse ist nach Budget-Erhöhung oder Monatsreset wieder möglich."- truncated: "Die Antwort der KI war zu lang und wurde abgeschnitten. Bitte versuchen Sie es erneut."+ truncated: "Die Antwort der KI war zu lang und wurde abgeschnitten. Bitte fordern Sie in der Arbeitsanweisung ein kürzeres Ergebnis an oder teilen Sie die Analyse auf."not_found: "Die Analyse existiert nicht mehr."prompt_not_found: "Die gewählte Analyse-Vorlage wurde nicht gefunden."prompt_blank: "Bitte eine Vorlage wählen oder einen Prompt eingeben."diff --git a/config/locales/services.en.yml b/config/locales/services.en.ymlindex dd5226c..dfda5bf 100644--- a/config/locales/services.en.yml+++ b/config/locales/services.en.yml@@ -131,7 +131,7 @@ en:empty_result: "The AI returned no result. Please try again."generic_error: "The analysis could not be created. Please try again later."quota_exhausted: "The AI quota is used up. The analysis is possible again after a budget increase or the monthly reset."- truncated: "The AI answer was too long and was cut off. Please try again."+ truncated: "The AI answer was too long and was cut off. Please ask for a shorter result in the instruction, or split the analysis."not_found: "The analysis no longer exists."prompt_not_found: "The selected analysis template was not found."prompt_blank: "Please choose a template or enter a prompt."diff --git a/spec/services/lalabase/llm/anthropic_provider_spec.rb b/spec/services/lalabase/llm/anthropic_provider_spec.rbindex de1f74c..4676126 100644--- a/spec/services/lalabase/llm/anthropic_provider_spec.rb+++ b/spec/services/lalabase/llm/anthropic_provider_spec.rb@@ -249,13 +249,16 @@ RSpec.describe Lalabase::Llm::AnthropicProvider do}end- it 'raises TruncatedError carrying the usage when max_tokens stopped it before any text' do- cut_response = Struct.new(:content, :usage, :stop_reason).new([], usage, 'max_tokens')- allow(fake_messages).to receive(:create).and_return(cut_response)- expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::TruncatedError) { |error|- expect(error.result).to include(content: '', stop_reason: 'length')- expect(error.result[:token_usage]).to be_present- }+ # The SDK reports stop_reason as a Symbol; a String is accepted as well.+ [:max_tokens, 'max_tokens'].each do |stop_reason|+ it "raises TruncatedError carrying the real usage when #{stop_reason.inspect} stopped it before any text" do+ cut_response = Struct.new(:content, :usage, :stop_reason).new([], usage(input: 37, output: 0), stop_reason)+ allow(fake_messages).to receive(:create).and_return(cut_response)+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::TruncatedError) { |error|+ expect(error.result).to include(content: '', stop_reason: 'length')+ expect(error.result[:token_usage]).to include(input_tokens: 37, output_tokens: 0, total_tokens: 37)+ }+ endendit 'maps Anthropic::Errors::AuthenticationError → AuthError' dodiff --git a/spec/services/lalabase/llm/open_ai_provider_spec.rb b/spec/services/lalabase/llm/open_ai_provider_spec.rbindex e93ff93..c9b9aff 100644--- a/spec/services/lalabase/llm/open_ai_provider_spec.rb+++ b/spec/services/lalabase/llm/open_ai_provider_spec.rb@@ -190,7 +190,23 @@ RSpec.describe Lalabase::Llm::OpenAiProvider doit 'raises an explicit Error when content is missing' doallow(fake_client).to receive(:chat).and_return({ 'choices' => [{ 'message' => { 'content' => nil } }] })- expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error, /empty response/)+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error, /empty response/) { |error|+ expect(error).not_to be_a(Lalabase::Llm::Provider::TruncatedError)+ }+ end++ # A reasoning model (or a compatible endpoint) can stop at the limit with+ # content null: same contract as Vertex and Anthropic, see TruncatedError.+ it 'raises TruncatedError carrying the usage when finish_reason length left content null' do+ allow(fake_client).to receive(:chat).and_return(+ { 'choices' => [{ 'message' => { 'content' => nil }, 'finish_reason' => 'length' }],+ 'usage' => { 'prompt_tokens' => 21, 'completion_tokens' => 5596, 'total_tokens' => 5617 } }+ )+ expect { provider.chat(messages: messages, model: 'gpt-5') }+ .to raise_error(Lalabase::Llm::Provider::TruncatedError, /empty response/) { |error|+ expect(error.result).to include(content: '', stop_reason: 'length', model_used: 'gpt-5')+ expect(error.result[:token_usage]).to include(input_tokens: 21, output_tokens: 5596, total_tokens: 5617)+ }endit 'maps Faraday::UnauthorizedError → AuthError' dodiff --git a/spec/services/lalabase/llm/provider_spec.rb b/spec/services/lalabase/llm/provider_spec.rbindex 3794cb9..cf942c8 100644--- a/spec/services/lalabase/llm/provider_spec.rb+++ b/spec/services/lalabase/llm/provider_spec.rb@@ -5,6 +5,17 @@ require 'rails_helper'RSpec.describe Lalabase::Llm::Provider dosubject(:provider) { described_class.new(api_key: 'k') }+ describe Lalabase::Llm::Provider::TruncatedError do+ it 'is a Provider::Error, so every caller that rescues Error behaves as before' do+ expect(described_class.ancestors).to include(Lalabase::Llm::Provider::Error)+ end++ it 'always carries a Hash as result, also when raised without one or with nil' do+ expect(described_class.new('cut').result).to eq({})+ expect(described_class.new('cut', result: nil).result).to eq({})+ end+ end+describe '#chat / #stream_chat / #embed' doit 'raises NotImplementedError on chat' doexpect { provider.chat(messages: []) }.to raise_error(NotImplementedError)diff --git a/spec/services/meetings/analysis_service_spec.rb b/spec/services/meetings/analysis_service_spec.rbindex d53a4c7..c1cac53 100644--- a/spec/services/meetings/analysis_service_spec.rb+++ b/spec/services/meetings/analysis_service_spec.rb@@ -196,6 +196,31 @@ RSpec.describe Meetings::AnalysisService doendend+ describe 'TruncatedError raised without a result' do+ let(:transcript) { 'Some transcript' }++ before { allow(provider).to receive(:chat).and_raise(Lalabase::Llm::Provider::TruncatedError.new('cut', result: nil)) }++ it 'still records the truncated error and clears the flag instead of raising' do+ expect { described_class.call(analysis: analysis) }.not_to raise_error+ analysis.reload+ expect(analysis.creation_error).to eq(I18n.t('services.meetings.analysis.truncated'))+ expect(analysis.creation_flag).to be(false)+ end+ end++ # A protocol of a long meeting does not fit into 1,500 tokens. With a cut+ # answer now being an error, a model without a thinking reserve (Mistral,+ # gpt-4o-mini) would otherwise fail every long analysis the same way.+ describe 'answer budget' do+ let(:transcript) { 'Some transcript' }++ it 'asks for up to 8,000 answer tokens' do+ described_class.call(analysis: analysis)+ expect(provider).to have_received(:chat).with(hash_including(max_tokens: 8_000))+ end+ end+describe 'an answer that ended normally (stop_reason stop)' dolet(:transcript) { 'Some transcript' }let(:chat_result) { { content: 'Fertige Analyse.', stop_reason: 'stop', token_usage: {}, model_used: 'gpt-4o-mini' } }diff --git a/spec/services/meetings/ticket_suggestion_service_spec.rb b/spec/services/meetings/ticket_suggestion_service_spec.rbindex 0d17c32..e9e7833 100644--- a/spec/services/meetings/ticket_suggestion_service_spec.rb+++ b/spec/services/meetings/ticket_suggestion_service_spec.rb@@ -211,7 +211,9 @@ RSpec.describe Meetings::TicketSuggestionService doendend- context 'when the answer of an assistant-started run was cut at the output limit' do+ # A run started from the meeting page (no chat post, no pinned sources). A+ # run from the assistant goes through the same record_error, see mark_failed!.+ context 'when the answer of a run was cut at the output limit' dolet(:chat_result) { { content: llm_content, stop_reason: 'length', token_usage: { total_tokens: 100 }, model_used: 'gpt-4o-mini' } }let(:run) doMeetingTicketSuggestionRun.create!(