A programming framework for agentic AI
More documentation for Cache. Updated FAQ to include database Locked error (#1291)
Balaca56782a7f63
diff --git a/autogen/oai/__init__.py b/autogen/oai/__init__.pyindex dbcd2f796..92791fd5c 100644--- a/autogen/oai/__init__.py+++ b/autogen/oai/__init__.py@@ -8,6 +8,7 @@ from autogen.oai.openai_utils import (config_list_from_json,config_list_from_dotenv,)+from autogen.cache.cache import Cache__all__ = ["OpenAIWrapper",@@ -19,4 +20,5 @@ __all__ = ["config_list_from_models","config_list_from_json","config_list_from_dotenv",+ "Cache",]diff --git a/website/docs/FAQ.md b/website/docs/FAQ.mdindex f6df3c236..442399f02 100644--- a/website/docs/FAQ.md+++ b/website/docs/FAQ.md@@ -192,6 +192,34 @@ Please refer to https://microsoft.github.io/autogen/docs/reference/agentchat/conThe "use_docker" arg in an agent's code_execution_config will be set to the name of the image containing the change after execution, when the conversation finishes.You can save that image name. For a new conversation, you can set "use_docker" to the saved name of the image to start execution there.+## Database locked error++When using VMs such as Azure Machine Learning compute instances,+you may encounter a "database locked error". This is because the+[LLM cache](./Use-Cases/agent_chat.md#cache)+is trying to write to a location that the application does not have access to.++You can set the `cache_path_root` to a location where the application has access.+For example,++```python+from autogen import Cache++with Cache.disk(cache_path_root="/tmp/.cache") as cache:+ agent_a.initate_chat(agent_b, ..., cache=cache)+```++You can also use Redis cache instead of disk cache. For example,++```python+from autogen import Cache++with Cache.redis(redis_url=...) as cache:+ agent_a.initate_chat(agent_b, ..., cache=cache)+```++You can also disable the cache. See [here](./Use-Cases/agent_chat.md#llm-caching) for details.+## Agents are throwing due to docker not running, how can I resolve this?If running AutoGen locally the default for agents who execute code is for them to try and perform code execution within a docker container. If docker is not running, this will cause the agent to throw an error. To resolve this you have the below options:diff --git a/website/docs/Use-Cases/agent_chat.md b/website/docs/Use-Cases/agent_chat.mdindex 98c103aa4..20965271a 100644--- a/website/docs/Use-Cases/agent_chat.md+++ b/website/docs/Use-Cases/agent_chat.md@@ -287,15 +287,37 @@ By adopting the conversation-driven control with both programming language and n### LLM Caching-Since version 0.2.8, a configurable context manager allows you to easily configure LLM cache, using either DiskCache or Redis. All agents inside the context manager will use the same cache.+Since version 0.2.8, a configurable context manager allows you to easily+configure LLM cache, using either DiskCache or Redis. All agents inside the+context manager will use the same cache.```python-from autogen.cache.cache import Cache+from autogen import Cache-with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache:+# Use Redis as cache+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:user.initiate_chat(assistant, message=coding_task, cache=cache)-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:+# Use DiskCache as cache+with Cache.disk() as cache:+ user.initiate_chat(assistant, message=coding_task, cache=cache)+```++You can vary the `cache_seed` parameter to get different LLM output while+still using cache.++```python+# Setting the cache_seed to 1 will use a different cache from the default one+# and you will see different output.+with Cache.disk(cache_seed=1) as cache:+ user.initiate_chat(assistant, message=coding_task, cache=cache)+```++By default DiskCache uses `.cache` for storage. To change the cache directory,+set `cache_path_root`:++```python+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:user.initiate_chat(assistant, message=coding_task, cache=cache)```diff --git a/website/docs/Use-Cases/enhanced_inference.md b/website/docs/Use-Cases/enhanced_inference.mdindex 17f871e46..e81c7281e 100644--- a/website/docs/Use-Cases/enhanced_inference.md+++ b/website/docs/Use-Cases/enhanced_inference.md@@ -168,20 +168,22 @@ Total cost: 0.00027## Caching-API call results are cached locally and reused when the same request is issued. This is useful when repeating or continuing experiments for reproducibility and cost saving.+API call results are cached locally and reused when the same request is issued.+This is useful when repeating or continuing experiments for reproducibility and cost saving.Starting version 0.2.8, a configurable context manager allows you to easily configurethe cache, using either DiskCache or Redis.-All `OpenAIWrapper` created inside the context manager can use the same cache through the constructor.+All `OpenAIWrapper` created inside the context manager can use the same cache+through the constructor.```python-from autogen.cache.cache import Cache+from autogen import Cache-with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache:+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:client = OpenAIWrapper(..., cache=cache)client.create(...)-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:+with Cache.disk() as cache:client = OpenAIWrapper(..., cache=cache)client.create(...)```@@ -189,12 +191,28 @@ with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:You can also set a cache directly in the `create()` method.```python-client = OpenAIWrapper()-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:+client = OpenAIWrapper(...)+with Cache.disk() as cache:+ client.create(..., cache=cache)+```++You can vary the `cache_seed` parameter to get different LLM output while+still using cache.++```python+# Setting the cache_seed to 1 will use a different cache from the default one+# and you will see different output.+with Cache.disk(cache_seed=1) as cache:client.create(..., cache=cache)```-You can control the randomness by setting the `cache_seed` parameter.+By default DiskCache uses `.cache` for storage. To change the cache directory,+set `cache_path_root`:++```python+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:+ client.create(..., cache=cache)+```### Turnning off cache@@ -208,9 +226,16 @@ client = OpenAIWrapper(..., cache_seed=None)client.create(..., cache_seed=None)```-_NOTE_. openai v1.1 introduces a new param `seed`. The difference between autogen's `cache_seed` and openai's `seed` is that:-* autogen uses local disk cache to guarantee the exactly same output is produced for the same input and when cache is hit, no openai api call will be made.-* openai's `seed` is a best-effort deterministic sampling with no guarantee of determinism. When using openai's `seed` with `cache_seed` set to None, even for the same input, an openai api call will be made and there is no guarantee for getting exactly the same output.+### Difference between `cache_seed` and openai's `seed` parameter++openai v1.1 introduces a new param `seed`.+The differences between autogen's `cache_seed` and openai's `seed`:+ - autogen uses local disk cache to guarantee the exactly same output is produced+ for the same input and when cache is hit, no openai api call will be made.+ - openai's `seed` is a best-effort deterministic sampling with no guarantee+ of determinism. When using openai's `seed` with `cache_seed` set to None,+ even for the same input, an openai api call will be made and there is+ no guarantee for getting exactly the same output.## Error handling