💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
Python agentsaiai-agentsembeddingsinformation-retrievallanguage-modellarge-language-modelsllmnlppythonragretrieval-augmented-generationsearchsearch-enginesemantic-searchsentence-embeddingstransformerstxtaivector-databasevector-search
Add clarification on building word vector backed models with scoring #247
davidmezzettifa342a4f6a69
diff --git a/docs/embeddings/index.md b/docs/embeddings/index.mdindex 02d712f4..da14fdf7 100644--- a/docs/embeddings/index.md+++ b/docs/embeddings/index.md@@ -71,6 +71,16 @@ The input iterable can be a list and/or a generator. [Generators](https://wiki.ptxtai buffers data to temporary storage along the way during indexing as embeddings vectors can be quite large (for example 768 dimensions of float32 is 768 * 4 = 3072 bytes per vector).+### Scoring+When using [word vector backed models](./configuration#words) with scoring set, a separate call is required before calling `index` as follows:++```python+embeddings.score([(uid, text, None) for uid, text in enumerate(data)])+embeddings.index([(uid, text, None) for uid, text in enumerate(data)])+```++Two calls are required to support generator-backed iteration of data. The scoring index requires a separate full-pass of the data.+## SearchOnce data is indexed, it is ready for search.