The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
Documentation Update for Upstash Semantic Caching (#25114)
Eray Eroğlua2e9910268d9
diff --git a/docs/docs/integrations/llm_caching.ipynb b/docs/docs/integrations/llm_caching.ipynbindex 4ac832199e..72d058e296 100644--- a/docs/docs/integrations/llm_caching.ipynb+++ b/docs/docs/integrations/llm_caching.ipynb@@ -334,6 +334,121 @@"llm.invoke(\"Tell me a joke\")"]},+ {+ "cell_type": "markdown",+ "id": "b29dd776",+ "metadata": {},+ "source": [+ "### Semantic Cache\n",+ "Use [Upstash Vector](https://upstash.com/docs/vector/overall/whatisvector) to do a semantic similarity search and cache the most similar response in the database. The vectorization is automatically done by the selected embedding model while creating Upstash Vector database. "+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "b37fb3c9",+ "metadata": {},+ "outputs": [],+ "source": [+ "%pip install upstash-semantic-cache"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 11,+ "id": "8470eedc",+ "metadata": {},+ "outputs": [],+ "source": [+ "from langchain.globals import set_llm_cache\n",+ "from upstash_semantic_cache import SemanticCache"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 12,+ "id": "16b9fb03",+ "metadata": {},+ "outputs": [],+ "source": [+ "UPSTASH_VECTOR_REST_URL = \"<UPSTASH_VECTOR_REST_URL>\"\n",+ "UPSTASH_VECTOR_REST_TOKEN = \"<UPSTASH_VECTOR_REST_TOKEN>\"\n",+ "\n",+ "cache = SemanticCache(\n",+ " url=UPSTASH_VECTOR_REST_URL, token=UPSTASH_VECTOR_REST_TOKEN, min_proximity=0.7\n",+ ")"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 15,+ "id": "8d37104b",+ "metadata": {},+ "outputs": [],+ "source": [+ "set_llm_cache(cache)"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 16,+ "id": "926a08b3",+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "CPU times: user 28.4 ms, sys: 3.93 ms, total: 32.3 ms\n",+ "Wall time: 1.89 s\n"+ ]+ },+ {+ "data": {+ "text/plain": [+ "'\\n\\nNew York City is the most crowded city in the USA.'"+ ]+ },+ "execution_count": 16,+ "metadata": {},+ "output_type": "execute_result"+ }+ ],+ "source": [+ "%%time\n",+ "llm.invoke(\"Which city is the most crowded city in the USA?\")"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 17,+ "id": "0ce37d57",+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "CPU times: user 3.22 ms, sys: 940 μs, total: 4.16 ms\n",+ "Wall time: 97.7 ms\n"+ ]+ },+ {+ "data": {+ "text/plain": [+ "'\\n\\nNew York City is the most crowded city in the USA.'"+ ]+ },+ "execution_count": 17,+ "metadata": {},+ "output_type": "execute_result"+ }+ ],+ "source": [+ "%%time\n",+ "llm.invoke(\"Which city has the highest population in the USA?\")"+ ]+ },{"cell_type": "markdown","id": "278ad7ae",@@ -2684,7 +2799,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.10.5"+ "version": "3.12.3"}},"nbformat": 4,