Skip to content

Distribute and run LLMs with a single file.

Commits on main

Fix POSIX undefined cancelation behavior
Justine Tunney committed 2 years ago
8604e9a
Add barebones /completion endpoint to new server
Justine Tunney committed 2 years ago
fb3421c
Update Discord invite link
Stephen Hood committed 2 years ago
814780a
Make GGML asynchronously cancelable
Justine Tunney committed 2 years ago
b3930aa
Add support for JSON parameters to new server
Justine Tunney committed 2 years ago
d7c8e33
Revert "Disable warmup"
Justine Tunney committed 2 years ago
1601118
Disable warmup
Justine Tunney committed 2 years ago
21a30be
Release llamafile v0.8.9
Justine Tunney committed 2 years ago
cd84736
Make gemma2-27b-it the same as aistudio.google.com
Justine Tunney committed 2 years ago
af22695
Add sliding window mask for Gemma2
Xuan Son Nguyen committed 2 years ago
41678c8
Reclaim llama_decode() memory on cancelation
Justine Tunney committed 2 years ago
0d62d05
Remove ggml_context cache
Justine Tunney committed 2 years ago
617d841
Upgrade to Cosmopolitan v3.5.4
Justine Tunney committed 2 years ago
3af1ac0
Add soft-capping to Gemma2
Andrei committed 2 years ago
140eed5
Use float to string conversion
Justine Tunney committed 2 years ago
263d39b
Upgrade to Cosmopolitan v3.5.3
Justine Tunney committed 2 years ago
3fc00b1
Create /embedding endpoint in new server
Justine Tunney committed 2 years ago
1346ef4
Refactor new server and get leak checker working
Justine Tunney committed 2 years ago
46dda4f
Prevent vector overflow in llama.cpp
Justine Tunney committed 2 years ago
cd73243
Release llamafile v0.8.8
Justine Tunney committed 2 years ago
571b4e5
Support flash attention in --server mode
Justine Tunney committed 2 years ago
4aea606
Add Google Gemma v2 support
Justine Tunney committed 2 years ago
7692b85
Introduce --special flag
Justine Tunney committed 2 years ago
72fb8ca
Don't flush bf16 subnormals to zero
Justine Tunney committed 2 years ago
7fd9101
Release llamafile v0.8.7
Justine Tunney committed 2 years ago
b2f587c
Cut flash attention from CUDA again
Justine Tunney committed 2 years ago
4d1fde0
Fix server crash due to /dev/urandom
Justine Tunney committed 2 years ago
629e208
Upgrade to Cosmopolitan v3.5.1
Justine Tunney committed 2 years ago
0c0e72a
Pacify --temp flag when running in server mode
Justine Tunney committed 2 years ago
6d3590c
Always use tinyBLAS with AMD GPUs on Windows
Justine Tunney committed 2 years ago
60404a8