💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
Python agentsaiai-agentsembeddingsinformation-retrievallanguage-modellarge-language-modelsllmnlppythonragretrieval-augmented-generationsearchsearch-enginesemantic-searchsentence-embeddingstransformerstxtaivector-databasevector-search
Revert change limiting tokenizer version #253
davidmezzetti3e898a8bac1f
diff --git a/setup.py b/setup.pyindex 52125728..4b81cb15 100644--- a/setup.py+++ b/setup.py@@ -6,7 +6,7 @@ with open("README.md", "r", encoding="utf-8") as f:DESCRIPTION = "".join([line for line in f if "gh-dark-mode-only" not in line])# Required dependencies-install = ["faiss-cpu>=1.7.1.post2", "numpy>=1.18.4", "pyyaml>=5.3", "torch>=1.6.0", "transformers>=4.12.3", "tokenizers>=0.10.3,<0.12.0"]+install = ["faiss-cpu>=1.7.1.post2", "numpy>=1.18.4", "pyyaml>=5.3", "torch>=1.6.0", "transformers>=4.12.3", "tokenizers>=0.10.3"]# Optional dependenciesextras = {}