Skip to content

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

llama: add minimum memory for grammar (#10820)

Parth Sareen884d26093c80
diff --git a/llama/llama.go b/llama/llama.go
index adee6f63c..0dc64e57e 100644
--- a/llama/llama.go
+++ b/llama/llama.go
@@ -580,7 +580,7 @@ func SchemaToGrammar(schema []byte) []byte {
defer C.free(unsafe.Pointer(cStr))
// Allocate buffer for grammar based on schema length but with upper bound
- maxLen := min(1024*1024, len(schema)*4)
+ maxLen := max(32768, min(1024*1024, len(schema)*4))
buf := make([]byte, maxLen)
// Call C function to convert schema to grammar