Skip to content

Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.

Fix review round 2 of #49: keep long analyses working and tighten the truncation contract

Guido Boykecb03a6b86d41
diff --git a/app/services/lalabase/CLAUDE.md b/app/services/lalabase/CLAUDE.md
index 8f0b3e2..630b33b 100644
--- a/app/services/lalabase/CLAUDE.md
+++ b/app/services/lalabase/CLAUDE.md
@@ -235,7 +235,7 @@ Wichtig:
(etwa `reasoning_effort`) schickt nur ein Seam mit Registry-Pin (`:customer_description`).
Endet eine Antwort am Limit (`stop_reason 'length'`), ist sie abgeschnitten:
`Meetings::BaseLlmService` meldet dann den Fehler `truncated` statt den Teiltext zu
- speichern. Kam vor dem Limit gar kein Text, werfen Vertex und Anthropic
+ speichern. Kam vor dem Limit gar kein Text, werfen OpenAI, Vertex und Anthropic
`Provider::TruncatedError` (eine Unterklasse von `Provider::Error`, mit dem Ergebnis samt
Verbrauch); `BaseLlmService` bucht ihn und meldet ebenfalls `truncated`. ⚠️ Der Assistent
(`Tools::Runner`) und die Teilnehmer-Erkennung werten `'length'` noch nicht aus.
diff --git a/app/services/lalabase/llm/open_ai_provider.rb b/app/services/lalabase/llm/open_ai_provider.rb
index 4ba4ec0..0416b88 100644
--- a/app/services/lalabase/llm/open_ai_provider.rb
+++ b/app/services/lalabase/llm/open_ai_provider.rb
@@ -95,15 +95,22 @@ module Lalabase
content = msg['content']
raw_tool_calls = Array(msg['tool_calls'])
- raise Error, "OpenAI returned empty response: #{response.inspect}" if content.nil? && raw_tool_calls.empty?
-
- {
+ result = {
content: content.to_s.strip,
tool_calls: raw_tool_calls.empty? ? nil : raw_tool_calls.map { |tc| parse_tool_call(tc) },
stop_reason: normalise_stop_reason(response.dig('choices', 0, 'finish_reason')),
model_used: model,
token_usage: extract_token_usage(response, model: model)
}
+ if content.nil? && raw_tool_calls.empty?
+ message = "OpenAI returned empty response: #{response.inspect}"
+ # Stopped at the limit before any text: see Provider::TruncatedError.
+ raise TruncatedError.new(message, result: result) if result[:stop_reason] == 'length'
+
+ raise Error, message
+ end
+
+ result
rescue Faraday::UnauthorizedError, Faraday::TooManyRequestsError => e
raise mapped_error(e)
end
diff --git a/app/services/lalabase/llm/provider.rb b/app/services/lalabase/llm/provider.rb
index ce00613..9541182 100644
--- a/app/services/lalabase/llm/provider.rb
+++ b/app/services/lalabase/llm/provider.rb
@@ -42,7 +42,7 @@ module Lalabase
def initialize(message = nil, result: {})
super(message)
- @result = result
+ @result = result || {}
end
end
@@ -126,7 +126,8 @@ module Lalabase
# thinks first puts its own reserve on top (GoogleVertexProvider::THINKING_BUDGET,
# OpenAiProvider::REASONING_RESERVE); callers never add one themselves. An
# answer that hit the limit before any text raises TruncatedError (with the
- # result and its usage) where a provider would otherwise raise "empty content".
+ # result and its usage) in #chat of the OpenAI, Anthropic and Vertex
+ # providers, where they would otherwise raise "empty content".
# @param temperature [Float]
# @return [Hash] {content:, tool_calls: nil | Array, stop_reason: String,
# model_used:, token_usage: {...}}.
diff --git a/app/services/meetings/analysis_service.rb b/app/services/meetings/analysis_service.rb
index 3648ff7..4113482 100644
--- a/app/services/meetings/analysis_service.rb
+++ b/app/services/meetings/analysis_service.rb
@@ -16,9 +16,12 @@ module Meetings
# A literal 'gpt-4o-mini' would be a runtime 404 on a Mistral/IONOS endpoint,
# and the task profile is provider-neutral anyway: a prompt in, German prose
# out — every chat model can do that.
- # An analysis (protocol, decision list, redacted version) can run longer than a
- # 150-word summary; still bounded, and the SOURCE_CHAR_LIMIT cap bounds input.
- MAX_TOKENS = 1_500
+ # An analysis (protocol, decision list, redacted version) can run much longer
+ # than a 150-word summary: a protocol of a long meeting does not fit into
+ # 1,500 tokens. Since an answer cut at the limit is an error (BaseLlmService),
+ # a small budget would fail every long analysis on a model without a thinking
+ # reserve. Still bounded; SOURCE_CHAR_LIMIT bounds the input.
+ MAX_TOKENS = 8_000
TEMPERATURE = 0.5
def self.call(analysis:)
diff --git a/config/locales/services.de.yml b/config/locales/services.de.yml
index b1f88e3..1b0f7d6 100644
--- a/config/locales/services.de.yml
+++ b/config/locales/services.de.yml
@@ -131,7 +131,7 @@ de:
empty_result: "Die KI hat kein Ergebnis zurückgegeben. Bitte versuchen Sie es erneut."
generic_error: "Die Analyse konnte nicht erstellt werden. Bitte versuchen Sie es später erneut."
quota_exhausted: "Das KI-Kontingent ist aufgebraucht. Die Analyse ist nach Budget-Erhöhung oder Monatsreset wieder möglich."
- truncated: "Die Antwort der KI war zu lang und wurde abgeschnitten. Bitte versuchen Sie es erneut."
+ truncated: "Die Antwort der KI war zu lang und wurde abgeschnitten. Bitte fordern Sie in der Arbeitsanweisung ein kürzeres Ergebnis an oder teilen Sie die Analyse auf."
not_found: "Die Analyse existiert nicht mehr."
prompt_not_found: "Die gewählte Analyse-Vorlage wurde nicht gefunden."
prompt_blank: "Bitte eine Vorlage wählen oder einen Prompt eingeben."
diff --git a/config/locales/services.en.yml b/config/locales/services.en.yml
index dd5226c..dfda5bf 100644
--- a/config/locales/services.en.yml
+++ b/config/locales/services.en.yml
@@ -131,7 +131,7 @@ en:
empty_result: "The AI returned no result. Please try again."
generic_error: "The analysis could not be created. Please try again later."
quota_exhausted: "The AI quota is used up. The analysis is possible again after a budget increase or the monthly reset."
- truncated: "The AI answer was too long and was cut off. Please try again."
+ truncated: "The AI answer was too long and was cut off. Please ask for a shorter result in the instruction, or split the analysis."
not_found: "The analysis no longer exists."
prompt_not_found: "The selected analysis template was not found."
prompt_blank: "Please choose a template or enter a prompt."
diff --git a/spec/services/lalabase/llm/anthropic_provider_spec.rb b/spec/services/lalabase/llm/anthropic_provider_spec.rb
index de1f74c..4676126 100644
--- a/spec/services/lalabase/llm/anthropic_provider_spec.rb
+++ b/spec/services/lalabase/llm/anthropic_provider_spec.rb
@@ -249,13 +249,16 @@ RSpec.describe Lalabase::Llm::AnthropicProvider do
}
end
- it 'raises TruncatedError carrying the usage when max_tokens stopped it before any text' do
- cut_response = Struct.new(:content, :usage, :stop_reason).new([], usage, 'max_tokens')
- allow(fake_messages).to receive(:create).and_return(cut_response)
- expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::TruncatedError) { |error|
- expect(error.result).to include(content: '', stop_reason: 'length')
- expect(error.result[:token_usage]).to be_present
- }
+ # The SDK reports stop_reason as a Symbol; a String is accepted as well.
+ [:max_tokens, 'max_tokens'].each do |stop_reason|
+ it "raises TruncatedError carrying the real usage when #{stop_reason.inspect} stopped it before any text" do
+ cut_response = Struct.new(:content, :usage, :stop_reason).new([], usage(input: 37, output: 0), stop_reason)
+ allow(fake_messages).to receive(:create).and_return(cut_response)
+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::TruncatedError) { |error|
+ expect(error.result).to include(content: '', stop_reason: 'length')
+ expect(error.result[:token_usage]).to include(input_tokens: 37, output_tokens: 0, total_tokens: 37)
+ }
+ end
end
it 'maps Anthropic::Errors::AuthenticationError → AuthError' do
diff --git a/spec/services/lalabase/llm/open_ai_provider_spec.rb b/spec/services/lalabase/llm/open_ai_provider_spec.rb
index e93ff93..c9b9aff 100644
--- a/spec/services/lalabase/llm/open_ai_provider_spec.rb
+++ b/spec/services/lalabase/llm/open_ai_provider_spec.rb
@@ -190,7 +190,23 @@ RSpec.describe Lalabase::Llm::OpenAiProvider do
it 'raises an explicit Error when content is missing' do
allow(fake_client).to receive(:chat).and_return({ 'choices' => [{ 'message' => { 'content' => nil } }] })
- expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error, /empty response/)
+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error, /empty response/) { |error|
+ expect(error).not_to be_a(Lalabase::Llm::Provider::TruncatedError)
+ }
+ end
+
+ # A reasoning model (or a compatible endpoint) can stop at the limit with
+ # content null: same contract as Vertex and Anthropic, see TruncatedError.
+ it 'raises TruncatedError carrying the usage when finish_reason length left content null' do
+ allow(fake_client).to receive(:chat).and_return(
+ { 'choices' => [{ 'message' => { 'content' => nil }, 'finish_reason' => 'length' }],
+ 'usage' => { 'prompt_tokens' => 21, 'completion_tokens' => 5596, 'total_tokens' => 5617 } }
+ )
+ expect { provider.chat(messages: messages, model: 'gpt-5') }
+ .to raise_error(Lalabase::Llm::Provider::TruncatedError, /empty response/) { |error|
+ expect(error.result).to include(content: '', stop_reason: 'length', model_used: 'gpt-5')
+ expect(error.result[:token_usage]).to include(input_tokens: 21, output_tokens: 5596, total_tokens: 5617)
+ }
end
it 'maps Faraday::UnauthorizedError → AuthError' do
diff --git a/spec/services/lalabase/llm/provider_spec.rb b/spec/services/lalabase/llm/provider_spec.rb
index 3794cb9..cf942c8 100644
--- a/spec/services/lalabase/llm/provider_spec.rb
+++ b/spec/services/lalabase/llm/provider_spec.rb
@@ -5,6 +5,17 @@ require 'rails_helper'
RSpec.describe Lalabase::Llm::Provider do
subject(:provider) { described_class.new(api_key: 'k') }
+ describe Lalabase::Llm::Provider::TruncatedError do
+ it 'is a Provider::Error, so every caller that rescues Error behaves as before' do
+ expect(described_class.ancestors).to include(Lalabase::Llm::Provider::Error)
+ end
+
+ it 'always carries a Hash as result, also when raised without one or with nil' do
+ expect(described_class.new('cut').result).to eq({})
+ expect(described_class.new('cut', result: nil).result).to eq({})
+ end
+ end
+
describe '#chat / #stream_chat / #embed' do
it 'raises NotImplementedError on chat' do
expect { provider.chat(messages: []) }.to raise_error(NotImplementedError)
diff --git a/spec/services/meetings/analysis_service_spec.rb b/spec/services/meetings/analysis_service_spec.rb
index d53a4c7..c1cac53 100644
--- a/spec/services/meetings/analysis_service_spec.rb
+++ b/spec/services/meetings/analysis_service_spec.rb
@@ -196,6 +196,31 @@ RSpec.describe Meetings::AnalysisService do
end
end
+ describe 'TruncatedError raised without a result' do
+ let(:transcript) { 'Some transcript' }
+
+ before { allow(provider).to receive(:chat).and_raise(Lalabase::Llm::Provider::TruncatedError.new('cut', result: nil)) }
+
+ it 'still records the truncated error and clears the flag instead of raising' do
+ expect { described_class.call(analysis: analysis) }.not_to raise_error
+ analysis.reload
+ expect(analysis.creation_error).to eq(I18n.t('services.meetings.analysis.truncated'))
+ expect(analysis.creation_flag).to be(false)
+ end
+ end
+
+ # A protocol of a long meeting does not fit into 1,500 tokens. With a cut
+ # answer now being an error, a model without a thinking reserve (Mistral,
+ # gpt-4o-mini) would otherwise fail every long analysis the same way.
+ describe 'answer budget' do
+ let(:transcript) { 'Some transcript' }
+
+ it 'asks for up to 8,000 answer tokens' do
+ described_class.call(analysis: analysis)
+ expect(provider).to have_received(:chat).with(hash_including(max_tokens: 8_000))
+ end
+ end
+
describe 'an answer that ended normally (stop_reason stop)' do
let(:transcript) { 'Some transcript' }
let(:chat_result) { { content: 'Fertige Analyse.', stop_reason: 'stop', token_usage: {}, model_used: 'gpt-4o-mini' } }
diff --git a/spec/services/meetings/ticket_suggestion_service_spec.rb b/spec/services/meetings/ticket_suggestion_service_spec.rb
index 0d17c32..e9e7833 100644
--- a/spec/services/meetings/ticket_suggestion_service_spec.rb
+++ b/spec/services/meetings/ticket_suggestion_service_spec.rb
@@ -211,7 +211,9 @@ RSpec.describe Meetings::TicketSuggestionService do
end
end
- context 'when the answer of an assistant-started run was cut at the output limit' do
+ # A run started from the meeting page (no chat post, no pinned sources). A
+ # run from the assistant goes through the same record_error, see mark_failed!.
+ context 'when the answer of a run was cut at the output limit' do
let(:chat_result) { { content: llm_content, stop_reason: 'length', token_usage: { total_tokens: 100 }, model_used: 'gpt-4o-mini' } }
let(:run) do
MeetingTicketSuggestionRun.create!(