Skip to content

A programming framework for agentic AI

Add Model Client Cache section to migration guide (#5197)

Eric Zhu138913bd5b7c
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/migration-guide.md b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/migration-guide.md
index b7a24777d..c88468595 100644
--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/migration-guide.md
+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/migration-guide.md
@@ -35,6 +35,7 @@ See each feature below for detailed information on how to migrate.
- [Model Client](#model-client)
- [Model Client for OpenAI-Compatible APIs](#model-client-for-openai-compatible-apis)
+- [Model Client Cache](#model-client-cache)
- [Assistant Agent](#assistant-agent)
- [Multi-Modal Agent](#multi-modal-agent)
- [User Proxy](#user-proxy)
@@ -59,8 +60,6 @@ See each feature below for detailed information on how to migrate.
The following features currently in `v0.2`
will be provided in the future releases of `v0.4.*` versions:
-- Model Client Cache [#4752](https://github.com/microsoft/autogen/issues/4752)
-- Jupyter Code Executor [#4795](https://github.com/microsoft/autogen/issues/4795)
- Model Client Cost [#4835](https://github.com/microsoft/autogen/issues/4835)
- Teachable Agent
- RAG Agent
@@ -161,6 +160,65 @@ in AgentChat Tutorial and more detailed information on [Core API Docs](../core-u
Support for other hosted models will be added in the future.
+## Model Client Cache
+
+In `v0.2`, you can set the cache seed through the `cache_seed` parameter in the LLM config.
+The cache is enabled by default.
+
+```python
+llm_config = {
+ "config_list": [{"model": "gpt-4o", "api_key": "sk-xxx"}],
+ "seed": 42,
+ "temperature": 0,
+ "cache_seed": 42,
+}
+```
+
+In `v0.4`, the cache is not enabled by default, to use it you need to use a
+{py:class}`~autogen_ext.models.cache.ChatCompletionCache` wrapper around the model client.
+
+You can use a {py:class}`~autogen_ext.cache_store.diskcache.DiskCacheStore` or {py:class}`~autogen_ext.cache_store.redis.RedisStore` to store the cache.
+
+```bash
+pip install -U "autogen-ext[openai, diskcache, redis]"
+```
+
+Here's an example of using `diskcache` for local caching:
+
+```python
+import asyncio
+import tempfile
+
+from autogen_core.models import UserMessage
+from autogen_ext.models.openai import OpenAIChatCompletionClient
+from autogen_ext.models.cache import ChatCompletionCache, CHAT_CACHE_VALUE_TYPE
+from autogen_ext.cache_store.diskcache import DiskCacheStore
+from diskcache import Cache
+
+
+async def main():
+ with tempfile.TemporaryDirectory() as tmpdirname:
+ # Initialize the original client
+ openai_model_client = OpenAIChatCompletionClient(model="gpt-4o")
+
+ # Then initialize the CacheStore, in this case with diskcache.Cache.
+ # You can also use redis like:
+ # from autogen_ext.cache_store.redis import RedisStore
+ # import redis
+ # redis_instance = redis.Redis()
+ # cache_store = RedisCacheStore[CHAT_CACHE_VALUE_TYPE](redis_instance)
+ cache_store = DiskCacheStore[CHAT_CACHE_VALUE_TYPE](Cache(tmpdirname))
+ cache_client = ChatCompletionCache(openai_model_client, cache_store)
+
+ response = await cache_client.create([UserMessage(content="Hello, how are you?", source="user")])
+ print(response) # Should print response from OpenAI
+ response = await cache_client.create([UserMessage(content="Hello, how are you?", source="user")])
+ print(response) # Should print cached response
+
+
+asyncio.run(main())
+```
+
## Assistant Agent
In `v0.2`, you create an assistant agent as follows: