A programming framework for agentic AI
Move LLM Caching docs to topics (#1950)
Jack Gerrits80233709e160
diff --git a/website/docs/Use-Cases/agent_chat.md b/website/docs/Use-Cases/agent_chat.mdindex 25664656c..921af166f 100644--- a/website/docs/Use-Cases/agent_chat.md+++ b/website/docs/Use-Cases/agent_chat.md@@ -327,56 +327,6 @@ With the pluggable auto-reply function, one can choose to invoke conversations wAnother approach involves LLM-based function calls, where LLM decides if a specific function should be invoked based on the conversation's status during each inference. This approach enables dynamic multi-agent conversations, as seen in scenarios like [multi-user math problem solving scenario](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_two_users.ipynb), where a student assistant automatically seeks expertise via function calls.-### LLM Caching--Since version 0.2.8, a configurable context manager allows you to easily-configure LLM cache, using either DiskCache or Redis. All agents inside the-context manager will use the same cache.--```python-from autogen import Cache--# Use Redis as cache-with Cache.redis(redis_url="redis://localhost:6379/0") as cache:- user.initiate_chat(assistant, message=coding_task, cache=cache)--# Use DiskCache as cache-with Cache.disk() as cache:- user.initiate_chat(assistant, message=coding_task, cache=cache)-```--You can vary the `cache_seed` parameter to get different LLM output while-still using cache.--```python-# Setting the cache_seed to 1 will use a different cache from the default one-# and you will see different output.-with Cache.disk(cache_seed=1) as cache:- user.initiate_chat(assistant, message=coding_task, cache=cache)-```--By default DiskCache uses `.cache` for storage. To change the cache directory,-set `cache_path_root`:--```python-with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:- user.initiate_chat(assistant, message=coding_task, cache=cache)-```--For backward compatibility, DiskCache is on by default with `cache_seed` set to 41.-To disable caching completely, set `cache_seed` to `None` in the `llm_config` of the agent.--```python-assistant = AssistantAgent(- "coding_agent",- llm_config={- "cache_seed": None,- "config_list": OAI_CONFIG_LIST,- "max_tokens": 1024,- },-)-```-### Diverse Applications Implemented with AutoGenThe figure below shows six examples of applications built using AutoGen.diff --git a/website/docs/topics/llm-caching.md b/website/docs/topics/llm-caching.mdnew file mode 100644index 000000000..c7a80133a--- /dev/null+++ b/website/docs/topics/llm-caching.md@@ -0,0 +1,49 @@+# LLM Caching++Since version [`0.2.8`](https://github.com/microsoft/autogen/releases/tag/v0.2.8), a configurable context manager allows you to easily+configure LLM cache, using either [`DiskCache`](/docs/reference/cache/disk_cache#diskcache) or [`RedisCache`](/docs/reference/cache/redis_cache#rediscache). All agents inside the+context manager will use the same cache.++```python+from autogen import Cache++# Use Redis as cache+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:+ user.initiate_chat(assistant, message=coding_task, cache=cache)++# Use DiskCache as cache+with Cache.disk() as cache:+ user.initiate_chat(assistant, message=coding_task, cache=cache)+```++You can vary the `cache_seed` parameter to get different LLM output while+still using cache.++```python+# Setting the cache_seed to 1 will use a different cache from the default one+# and you will see different output.+with Cache.disk(cache_seed=1) as cache:+ user.initiate_chat(assistant, message=coding_task, cache=cache)+```++By default [`DiskCache`](/docs/reference/cache/disk_cache#diskcache) uses `.cache` for storage. To change the cache directory,+set `cache_path_root`:++```python+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:+ user.initiate_chat(assistant, message=coding_task, cache=cache)+```++For backward compatibility, [`DiskCache`](/docs/reference/cache/disk_cache#diskcache) is on by default with `cache_seed` set to 41.+To disable caching completely, set `cache_seed` to `None` in the `llm_config` of the agent.++```python+assistant = AssistantAgent(+ "coding_agent",+ llm_config={+ "cache_seed": None,+ "config_list": OAI_CONFIG_LIST,+ "max_tokens": 1024,+ },+)+```