Skip to content

The agent engineering platform.

Documentation Update for Upstash Semantic Caching (#25114)

Eray Eroğlua2e9910268d9
diff --git a/docs/docs/integrations/llm_caching.ipynb b/docs/docs/integrations/llm_caching.ipynb
index 4ac832199e..72d058e296 100644
--- a/docs/docs/integrations/llm_caching.ipynb
+++ b/docs/docs/integrations/llm_caching.ipynb
@@ -334,6 +334,121 @@
"llm.invoke(\"Tell me a joke\")"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "b29dd776",
+ "metadata": {},
+ "source": [
+ "### Semantic Cache\n",
+ "Use [Upstash Vector](https://upstash.com/docs/vector/overall/whatisvector) to do a semantic similarity search and cache the most similar response in the database. The vectorization is automatically done by the selected embedding model while creating Upstash Vector database. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b37fb3c9",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%pip install upstash-semantic-cache"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "id": "8470eedc",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from langchain.globals import set_llm_cache\n",
+ "from upstash_semantic_cache import SemanticCache"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "id": "16b9fb03",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "UPSTASH_VECTOR_REST_URL = \"<UPSTASH_VECTOR_REST_URL>\"\n",
+ "UPSTASH_VECTOR_REST_TOKEN = \"<UPSTASH_VECTOR_REST_TOKEN>\"\n",
+ "\n",
+ "cache = SemanticCache(\n",
+ " url=UPSTASH_VECTOR_REST_URL, token=UPSTASH_VECTOR_REST_TOKEN, min_proximity=0.7\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "id": "8d37104b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "set_llm_cache(cache)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "id": "926a08b3",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 28.4 ms, sys: 3.93 ms, total: 32.3 ms\n",
+ "Wall time: 1.89 s\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "'\\n\\nNew York City is the most crowded city in the USA.'"
+ ]
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "llm.invoke(\"Which city is the most crowded city in the USA?\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "id": "0ce37d57",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 3.22 ms, sys: 940 μs, total: 4.16 ms\n",
+ "Wall time: 97.7 ms\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "'\\n\\nNew York City is the most crowded city in the USA.'"
+ ]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "llm.invoke(\"Which city has the highest population in the USA?\")"
+ ]
+ },
{
"cell_type": "markdown",
"id": "278ad7ae",
@@ -2684,7 +2799,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.5"
+ "version": "3.12.3"
}
},
"nbformat": 4,