Skip to content

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

llm: raise token repeat limit to 100 and return error instead of silently closing (#18374)

frob4512d2b76dd9
diff --git a/llm/llama_server.go b/llm/llama_server.go
index fc3edfbbe..0f4019799 100644
--- a/llm/llama_server.go
+++ b/llm/llama_server.go
@@ -1700,9 +1700,9 @@ func (s *llamaServerRunner) Completion(ctx context.Context, req CompletionReques
lastToken = strings.TrimSpace(lsResp.Content)
tokenRepeat = 0
}
- if tokenRepeat > 30 {
+ if tokenRepeat > 100 {
slog.Debug("prediction aborted, token repeat limit reached")
- return ctx.Err()
+ return fmt.Errorf("prediction aborted, token repeat limit reached")
}
if lsResp.Content != "" && !lsResp.Stop {