Skip to content
Arm
Git
Search or jump to…
Ctrl K
Categories
Theme: System
Sign in
Get started
armgit
/
ollama
Public
Imported from GitHub
Fork
0
Watch
0
Star
0
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Go
deepseek
gemma
gemma3
glm
go
golang
gpt-oss
llama
llama3
llm
llms
minimax
mistral
ollama
qwen
Code
Commits
Issues
Pull requests
Wiki
Builds
Discussions
Projects
Insights
Search
Files
main
486 branches · 712 tags
Go to file
t
Code
ollama
/
mlxrunner
/
tokenizer
Name
Last commit message
Last commit date
..
testdata
GO
tokenizer_added.go
1.5 KB
GO
tokenizer_benchmark_test.go
4.5 KB
GO
tokenizer_bpe_spans_test.go
3.3 KB
GO
tokenizer_bpe.go
4.2 KB
GO
tokenizer_config_test.go
3.3 KB
GO
tokenizer_correctness_test.go
3.4 KB
GO
tokenizer_decode.go
1.2 KB
GO
tokenizer_encode.go
5.4 KB
GO
tokenizer_ggml_parity_test.go
5.1 KB
GO
tokenizer_load_test.go
3.8 KB
GO
tokenizer_load.go
11.9 KB
GO
tokenizer_normalization_test.go
11.6 KB
GO
tokenizer_pretokenizer_test.go
9.7 KB
GO
tokenizer_pretokenizer.go
8.5 KB
GO
tokenizer_reference_benchmark_test.go
1.3 KB
GO
tokenizer_reference_model_test.go
6.9 KB
GO
tokenizer_reference_test.go
21.3 KB
GO
tokenizer.go
3.1 KB