Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.
Ruby agplai-assistantdockerdsgvogdprhotwireissue-trackerjira-alternativeknowledge-basellmmcp-servermeeting-notesopen-sourceproject-managementredmine-alternativeruby-on-railsself-hostedticketingtime-tracking
Fix review round 1 of #49: report and meter an answer cut before any text
Guido Boykecf335b3b845d
diff --git a/app/services/customer_description_generator.rb b/app/services/customer_description_generator.rbindex 9cc1005..2ac094a 100644--- a/app/services/customer_description_generator.rb+++ b/app/services/customer_description_generator.rb@@ -267,7 +267,8 @@ class CustomerDescriptionGeneratormodel: MODEL,max_tokens: 1000,# gpt-5-nano is a reasoning model: the provider maps max_tokens →- # max_completion_tokens and omits temperature; reasoning_effort passes through+ # max_completion_tokens (plus OpenAiProvider::REASONING_RESERVE, so 1000 stays+ # the answer budget) and omits temperature; reasoning_effort passes through# provider_options so the "fast, direct" behaviour is preserved 1:1.provider_options: { reasoning_effort: 'minimal' })diff --git a/app/services/lalabase/CLAUDE.md b/app/services/lalabase/CLAUDE.mdindex 1ffcc56..8f0b3e2 100644--- a/app/services/lalabase/CLAUDE.md+++ b/app/services/lalabase/CLAUDE.md@@ -231,8 +231,11 @@ Wichtig:für Gemini 2.5 Flash und Pro (`thinkingConfig` + `maxOutputTokens`),`OpenAiProvider::REASONING_RESERVE` für gpt-5 und die o-Reihe (`max_completion_tokens`).Ohne Reserve zählten die Denk-Tokens gegen das Limit, und von einer kurzen Antwort blieb- fast nichts übrig. Ein Aufrufer rechnet nie selbst eine Reserve ein und- schickt keine Anbieter-Felder. Endet eine Antwort am Limit (`stop_reason 'length'`), ist- sie abgeschnitten: `Meetings::BaseLlmService` meldet dann den Fehler `truncated` statt- den Teiltext zu speichern. ⚠️ Der Assistent (`Tools::Runner`) und die Teilnehmer-Erkennung- werten `'length'` noch nicht aus.+ fast nichts übrig. Ein Aufrufer rechnet nie selbst eine Reserve ein; Anbieter-Felder+ (etwa `reasoning_effort`) schickt nur ein Seam mit Registry-Pin (`:customer_description`).+ Endet eine Antwort am Limit (`stop_reason 'length'`), ist sie abgeschnitten:+ `Meetings::BaseLlmService` meldet dann den Fehler `truncated` statt den Teiltext zu+ speichern. Kam vor dem Limit gar kein Text, werfen Vertex und Anthropic+ `Provider::TruncatedError` (eine Unterklasse von `Provider::Error`, mit dem Ergebnis samt+ Verbrauch); `BaseLlmService` bucht ihn und meldet ebenfalls `truncated`. ⚠️ Der Assistent+ (`Tools::Runner`) und die Teilnehmer-Erkennung werten `'length'` noch nicht aus.diff --git a/app/services/lalabase/llm/anthropic_provider.rb b/app/services/lalabase/llm/anthropic_provider.rbindex e185861..979e440 100644--- a/app/services/lalabase/llm/anthropic_provider.rb+++ b/app/services/lalabase/llm/anthropic_provider.rb@@ -37,15 +37,21 @@ module Lalabaseresponse = client.messages.create(params)text_parts, tool_calls = extract_content_blocks(response)content_text = text_parts.join.strip- raise Error, 'Anthropic returned empty content' if content_text.empty? && tool_calls.empty?-- {+ result = {content: content_text,tool_calls: tool_calls.empty? ? nil : tool_calls,stop_reason: extract_stop_reason(response),model_used: model,token_usage: extract_token_usage(response, model: model)}+ if content_text.empty? && tool_calls.empty?+ # Stopped at max_tokens before any text: see Provider::TruncatedError.+ raise TruncatedError.new('Anthropic returned empty content', result: result) if result[:stop_reason] == 'length'++ raise Error, 'Anthropic returned empty content'+ end++ resultrescue Anthropic::Errors::APIError => eraise mapped_error(e)enddiff --git a/app/services/lalabase/llm/google_vertex_provider.rb b/app/services/lalabase/llm/google_vertex_provider.rbindex 3e08648..f71c0c3 100644--- a/app/services/lalabase/llm/google_vertex_provider.rb+++ b/app/services/lalabase/llm/google_vertex_provider.rb@@ -66,11 +66,13 @@ module Lalabase# fixed thinking budget and the same reserve on top of the answer budget.# 2,048 lies inside both accepted ranges (Pro 128..32,768, it cannot turn# thinking off; Flash 1..24,576).- # Not matched on purpose: Flash-Lite (a budget would switch thinking ON),+ # Not matched on purpose: Flash-Lite (a budget would switch thinking on),# 2.0 Flash (no thinking) and Gemini 3 (thinkingLevel instead, and# thinkingBudget is deprecated there and can answer 400).THINKING_BUDGET = 2_048- THINKING_MODEL_PATTERN = /\Agemini-2\.5-(?:pro|flash(?!-lite))/+ # Also not matched: the image, speech (tts), live and native-audio variants+ # of 2.5, which do not think, and a longer word after the family name.+ THINKING_MODEL_PATTERN = /\Agemini-2\.5-(?:pro|flash)(?:\z|-(?!.*(?:lite|image|tts|live|audio)))/# Gemini finishReason → canonical vocabulary (see Provider#chat). The# whole safety family collapses onto content_filter; unknown values pass@@ -191,18 +193,24 @@ module Lalabase# finishReason is right here on the candidate. Only a truly empty result# (no content, no tools, no filter signal) is an error — now carrying the# finishReason for diagnosis.- if content_text.empty? && tool_calls.empty? && stop_reason != 'content_filter'- finish = candidate['finishReason'].inspect- raise Error, "Vertex AI returned empty content (finishReason: #{finish})"- end-- {+ result = {content: content_text,tool_calls: tool_calls.empty? ? nil : tool_calls,stop_reason: stop_reason,model_used: model,token_usage: usage_from_metadata(response['usageMetadata'], model: model)}+ if content_text.empty? && tool_calls.empty? && stop_reason != 'content_filter'+ finish = candidate['finishReason'].inspect+ message = "Vertex AI returned empty content (finishReason: #{finish})"+ # MAX_TOKENS with no text: the thoughts used the whole limit (the+ # thinking budget is a target, not a hard cap). See TruncatedError.+ raise TruncatedError.new(message, result: result) if stop_reason == 'length'++ raise Error, message+ end++ resultenddef stream_chat(messages:, model: nil, max_tokens: 1500, temperature: 0.7, provider_options: {}, &on_chunk)diff --git a/app/services/lalabase/llm/provider.rb b/app/services/lalabase/llm/provider.rbindex 62aee8c..ce00613 100644--- a/app/services/lalabase/llm/provider.rb+++ b/app/services/lalabase/llm/provider.rb@@ -32,6 +32,20 @@ module Lalabase# space's expected dimension — raised by #guard_dimensions (see there).class DimensionMismatchError < Error; end+ # The answer hit max_tokens before a single word of it was written: a thinking+ # model can spend the whole limit on its thoughts. Still an Error, so every+ # caller that rescues Error behaves as before. It carries the result hash of+ # #chat (empty content, stop_reason 'length', token_usage) because the call+ # was billed: Meetings::BaseLlmService meters it and reports 'truncated'.+ class TruncatedError < Error+ attr_reader :result++ def initialize(message = nil, result: {})+ super(message)+ @result = result+ end+ end+# Fallback embedding dimension when none is configured on the provider# (embedding_dims: nil). Historically the hard "1536-lane" invariant# (doc/embedding_provider_abstraction_plan.md); with v2 the dimension is a@@ -110,7 +124,9 @@ module Lalabase# @param model [String, nil] Provider-specific model id. Falls back to #default_chat_model.# @param max_tokens [Integer] Budget of the ANSWER. A provider whose model# thinks first puts its own reserve on top (GoogleVertexProvider::THINKING_BUDGET,- # OpenAiProvider::REASONING_RESERVE); callers never add one themselves.+ # OpenAiProvider::REASONING_RESERVE); callers never add one themselves. An+ # answer that hit the limit before any text raises TruncatedError (with the+ # result and its usage) where a provider would otherwise raise "empty content".# @param temperature [Float]# @return [Hash] {content:, tool_calls: nil | Array, stop_reason: String,# model_used:, token_usage: {...}}.diff --git a/app/services/meetings/base_llm_service.rb b/app/services/meetings/base_llm_service.rbindex bca5c52..66797f1 100644--- a/app/services/meetings/base_llm_service.rb+++ b/app/services/meetings/base_llm_service.rb@@ -72,6 +72,11 @@ module Meetingsreturn record_truncated(result) if result[:stop_reason] == TRUNCATED_STOP_REASONhandle_result(result)+ rescue Lalabase::Llm::Provider::TruncatedError => e+ # The limit was reached before any text: the provider raises instead of+ # returning, but the call was billed all the same. Same outcome as above.+ track_usage(provider, e.result)+ record_truncated(e.result)rescue Lalabase::Llm::Factory::MissingCredentialErrorrecord_error(t('not_configured'))rescue Lalabase::Llm::QuotaExceededErrordiff --git a/spec/services/lalabase/llm/anthropic_provider_spec.rb b/spec/services/lalabase/llm/anthropic_provider_spec.rbindex d2305a9..de1f74c 100644--- a/spec/services/lalabase/llm/anthropic_provider_spec.rb+++ b/spec/services/lalabase/llm/anthropic_provider_spec.rb@@ -244,7 +244,18 @@ RSpec.describe Lalabase::Llm::AnthropicProvider doit 'raises Error when content blocks contain no text' doempty_response = Struct.new(:content, :usage).new([], usage)allow(fake_messages).to receive(:create).and_return(empty_response)- expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error, /empty content/)+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error, /empty content/) { |error|+ expect(error).not_to be_a(Lalabase::Llm::Provider::TruncatedError)+ }+ end++ it 'raises TruncatedError carrying the usage when max_tokens stopped it before any text' do+ cut_response = Struct.new(:content, :usage, :stop_reason).new([], usage, 'max_tokens')+ allow(fake_messages).to receive(:create).and_return(cut_response)+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::TruncatedError) { |error|+ expect(error.result).to include(content: '', stop_reason: 'length')+ expect(error.result[:token_usage]).to be_present+ }endit 'maps Anthropic::Errors::AuthenticationError → AuthError' dodiff --git a/spec/services/lalabase/llm/google_vertex_provider_spec.rb b/spec/services/lalabase/llm/google_vertex_provider_spec.rbindex 185614c..868ed21 100644--- a/spec/services/lalabase/llm/google_vertex_provider_spec.rb+++ b/spec/services/lalabase/llm/google_vertex_provider_spec.rb@@ -359,9 +359,11 @@ RSpec.describe Lalabase::Llm::GoogleVertexProvider do)end- it 'matches a versioned name of a thinking family' do- provider.chat(messages: messages, model: 'gemini-2.5-flash-002')- expect(last_payload['generationConfig']).to include('thinkingConfig' => { 'thinkingBudget' => budget })+ ['gemini-2.5-flash-002', 'gemini-2.5-pro-preview-05-06', 'gemini-2.5-flash-preview-09-2025'].each do |model|+ it "matches the versioned name #{model} of a thinking family" do+ provider.chat(messages: messages, model: model)+ expect(last_payload['generationConfig']).to include('thinkingConfig' => { 'thinkingBudget' => budget })+ endendit 'keeps the budget inside the range both 2.5 families accept (Pro 128..32768, Flash 1..24576)' do@@ -372,7 +374,12 @@ RSpec.describe Lalabase::Llm::GoogleVertexProvider do# thinking ON for Flash-Lite and cost tokens nobody asked for. Gemini 3 takes# thinkingLevel instead (thinkingBudget is deprecated there and can 400), and# whether its thoughts count against maxOutputTokens is not documented yet.- ['gemini-2.5-flash-lite', 'gemini-2.0-flash', 'gemini-3-pro-preview', 'gemini-3-flash', 'some-tuned-model']+ # The image, speech and live variants of 2.5 do not think either (a+ # thinkingConfig can be refused there), and a longer word is not the family.+ ['gemini-2.5-flash-lite', 'gemini-2.5-flash-lite-preview-06-17', 'gemini-2.5-flash-image',+ 'gemini-2.5-flash-image-preview', 'gemini-2.5-flash-preview-tts', 'gemini-2.5-pro-preview-tts',+ 'gemini-2.5-flash-live-preview', 'gemini-2.5-flash-native-audio-dialog', 'gemini-2.5-flashy',+ 'gemini-2.0-flash', 'gemini-3-pro-preview', 'gemini-3-flash', 'some-tuned-model'].each do |model|it "sends #{model} the bare limit, byte for byte as before" doprovider.chat(messages: messages, model: model)@@ -452,6 +459,27 @@ RSpec.describe Lalabase::Llm::GoogleVertexProvider do.to raise_error(Lalabase::Llm::Provider::Error, /empty content.*MAX_TOKENS/m)end+ # Thinking can still spend the whole output limit (the budget is a target, not+ # a hard cap). The empty answer stays an Error for every caller, but a+ # TruncatedError that carries the billed usage, so a caller can meter it and+ # say "cut off" instead of "something went wrong".+ it 'raises TruncatedError carrying stop_reason and usage when MAX_TOKENS left no text' do+ stub_post(body: chat_response(parts: [], finish_reason: 'MAX_TOKENS',+ usage: { 'promptTokenCount' => 10, 'candidatesTokenCount' => 0,+ 'totalTokenCount' => 3558 }))+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::TruncatedError) { |error|+ expect(error.result).to include(content: '', stop_reason: 'length', model_used: 'gemini-2.5-flash')+ expect(error.result[:token_usage]).to include(input_tokens: 10, total_tokens: 3558)+ }+ end++ it 'keeps a plain Error (not TruncatedError) for an empty answer that did not hit the limit' do+ stub_post(body: chat_response(parts: [], finish_reason: 'STOP'))+ expect { provider.chat(messages: messages) }.to raise_error(Lalabase::Llm::Provider::Error) { |error|+ expect(error).not_to be_a(Lalabase::Llm::Provider::TruncatedError)+ }+ end+it 'returns content_filter (not an error) when an output-side safety block empties the content' do# Gemini reports finishReason SAFETY/RECITATION with NO parts on an# output-side policy block — surface the contract's content_filter so thediff --git a/spec/services/meetings/analysis_service_spec.rb b/spec/services/meetings/analysis_service_spec.rbindex 56321f1..d53a4c7 100644--- a/spec/services/meetings/analysis_service_spec.rb+++ b/spec/services/meetings/analysis_service_spec.rb@@ -173,6 +173,29 @@ RSpec.describe Meetings::AnalysisService doendend+ describe 'answer cut before any text (provider raises TruncatedError)' do+ let(:transcript) { 'Some transcript' }+ let(:cut_result) do+ { content: '', stop_reason: 'length', model_used: 'gemini-2.5-flash',+ token_usage: { input_tokens: 900, output_tokens: 0, total_tokens: 3548 } }+ end++ before do+ allow(provider).to receive(:chat)+ .and_raise(Lalabase::Llm::Provider::TruncatedError.new('empty content', result: cut_result))+ end++ it 'records the truncated error (not the generic one) and meters the billed call' do+ described_class.call(analysis: analysis)+ analysis.reload+ expect(analysis.creation_error).to eq(I18n.t('services.meetings.analysis.truncated'))+ expect(analysis.creation_flag).to be(false)+ expect(analysis.content).to eq('')+ expect(Lalabase::Llm::UsageTracking).to have_received(:record)+ .with(hash_including(token_usage: cut_result[:token_usage], subject: analysis))+ end+ end+describe 'an answer that ended normally (stop_reason stop)' dolet(:transcript) { 'Some transcript' }let(:chat_result) { { content: 'Fertige Analyse.', stop_reason: 'stop', token_usage: {}, model_used: 'gpt-4o-mini' } }diff --git a/spec/services/meetings/ticket_suggestion_service_spec.rb b/spec/services/meetings/ticket_suggestion_service_spec.rbindex 0cc34f9..0d17c32 100644--- a/spec/services/meetings/ticket_suggestion_service_spec.rb+++ b/spec/services/meetings/ticket_suggestion_service_spec.rb@@ -211,6 +211,26 @@ RSpec.describe Meetings::TicketSuggestionService doendend+ context 'when the answer of an assistant-started run was cut at the output limit' do+ let(:chat_result) { { content: llm_content, stop_reason: 'length', token_usage: { total_tokens: 100 }, model_used: 'gpt-4o-mini' } }+ let(:run) do+ MeetingTicketSuggestionRun.create!(+ organisation: organisation, meeting: meeting, project_id: meeting.project_id, status: :running,+ meeting_title: meeting.title, project_title: project.name, job_id: SecureRandom.uuid,+ results: [], sources: {}+ )+ end++ it 'fails the run with the truncated message and creates no suggestion' do+ expect { described_class.call(meeting: meeting, run: run) }+ .not_to(change { meeting.meeting_ticket_suggestions.count })++ expect(run.reload).to be_failed+ expect(run.error).to eq(I18n.t('services.meetings.ticket_suggestions.truncated'))+ expect(meeting.reload.suggestions_creation_flag).to be(false)+ end+ end+context 'when the quota is exhausted (Factory raises)' dobefore { allow(Lalabase::Llm::Factory).to receive(:for_feature).with(:meeting_ticket_suggestions, organisation).and_raise(Lalabase::Llm::QuotaExceededError) }