Skip to content

A programming framework for agentic AI

More documentation for Cache. Updated FAQ to include database Locked error (#1291)

Balaca56782a7f63
diff --git a/autogen/oai/__init__.py b/autogen/oai/__init__.py
index dbcd2f796..92791fd5c 100644
--- a/autogen/oai/__init__.py
+++ b/autogen/oai/__init__.py
@@ -8,6 +8,7 @@ from autogen.oai.openai_utils import (
config_list_from_json,
config_list_from_dotenv,
)
+from autogen.cache.cache import Cache
__all__ = [
"OpenAIWrapper",
@@ -19,4 +20,5 @@ __all__ = [
"config_list_from_models",
"config_list_from_json",
"config_list_from_dotenv",
+ "Cache",
]
diff --git a/website/docs/FAQ.md b/website/docs/FAQ.md
index f6df3c236..442399f02 100644
--- a/website/docs/FAQ.md
+++ b/website/docs/FAQ.md
@@ -192,6 +192,34 @@ Please refer to https://microsoft.github.io/autogen/docs/reference/agentchat/con
The "use_docker" arg in an agent's code_execution_config will be set to the name of the image containing the change after execution, when the conversation finishes.
You can save that image name. For a new conversation, you can set "use_docker" to the saved name of the image to start execution there.
+## Database locked error
+
+When using VMs such as Azure Machine Learning compute instances,
+you may encounter a "database locked error". This is because the
+[LLM cache](./Use-Cases/agent_chat.md#cache)
+is trying to write to a location that the application does not have access to.
+
+You can set the `cache_path_root` to a location where the application has access.
+For example,
+
+```python
+from autogen import Cache
+
+with Cache.disk(cache_path_root="/tmp/.cache") as cache:
+ agent_a.initate_chat(agent_b, ..., cache=cache)
+```
+
+You can also use Redis cache instead of disk cache. For example,
+
+```python
+from autogen import Cache
+
+with Cache.redis(redis_url=...) as cache:
+ agent_a.initate_chat(agent_b, ..., cache=cache)
+```
+
+You can also disable the cache. See [here](./Use-Cases/agent_chat.md#llm-caching) for details.
+
## Agents are throwing due to docker not running, how can I resolve this?
If running AutoGen locally the default for agents who execute code is for them to try and perform code execution within a docker container. If docker is not running, this will cause the agent to throw an error. To resolve this you have the below options:
diff --git a/website/docs/Use-Cases/agent_chat.md b/website/docs/Use-Cases/agent_chat.md
index 98c103aa4..20965271a 100644
--- a/website/docs/Use-Cases/agent_chat.md
+++ b/website/docs/Use-Cases/agent_chat.md
@@ -287,15 +287,37 @@ By adopting the conversation-driven control with both programming language and n
### LLM Caching
-Since version 0.2.8, a configurable context manager allows you to easily configure LLM cache, using either DiskCache or Redis. All agents inside the context manager will use the same cache.
+Since version 0.2.8, a configurable context manager allows you to easily
+configure LLM cache, using either DiskCache or Redis. All agents inside the
+context manager will use the same cache.
```python
-from autogen.cache.cache import Cache
+from autogen import Cache
-with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache:
+# Use Redis as cache
+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:
user.initiate_chat(assistant, message=coding_task, cache=cache)
-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
+# Use DiskCache as cache
+with Cache.disk() as cache:
+ user.initiate_chat(assistant, message=coding_task, cache=cache)
+```
+
+You can vary the `cache_seed` parameter to get different LLM output while
+still using cache.
+
+```python
+# Setting the cache_seed to 1 will use a different cache from the default one
+# and you will see different output.
+with Cache.disk(cache_seed=1) as cache:
+ user.initiate_chat(assistant, message=coding_task, cache=cache)
+```
+
+By default DiskCache uses `.cache` for storage. To change the cache directory,
+set `cache_path_root`:
+
+```python
+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:
user.initiate_chat(assistant, message=coding_task, cache=cache)
```
diff --git a/website/docs/Use-Cases/enhanced_inference.md b/website/docs/Use-Cases/enhanced_inference.md
index 17f871e46..e81c7281e 100644
--- a/website/docs/Use-Cases/enhanced_inference.md
+++ b/website/docs/Use-Cases/enhanced_inference.md
@@ -168,20 +168,22 @@ Total cost: 0.00027
## Caching
-API call results are cached locally and reused when the same request is issued. This is useful when repeating or continuing experiments for reproducibility and cost saving.
+API call results are cached locally and reused when the same request is issued.
+This is useful when repeating or continuing experiments for reproducibility and cost saving.
Starting version 0.2.8, a configurable context manager allows you to easily configure
the cache, using either DiskCache or Redis.
-All `OpenAIWrapper` created inside the context manager can use the same cache through the constructor.
+All `OpenAIWrapper` created inside the context manager can use the same cache
+through the constructor.
```python
-from autogen.cache.cache import Cache
+from autogen import Cache
-with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache:
+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:
client = OpenAIWrapper(..., cache=cache)
client.create(...)
-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
+with Cache.disk() as cache:
client = OpenAIWrapper(..., cache=cache)
client.create(...)
```
@@ -189,12 +191,28 @@ with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
You can also set a cache directly in the `create()` method.
```python
-client = OpenAIWrapper()
-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
+client = OpenAIWrapper(...)
+with Cache.disk() as cache:
+ client.create(..., cache=cache)
+```
+
+You can vary the `cache_seed` parameter to get different LLM output while
+still using cache.
+
+```python
+# Setting the cache_seed to 1 will use a different cache from the default one
+# and you will see different output.
+with Cache.disk(cache_seed=1) as cache:
client.create(..., cache=cache)
```
-You can control the randomness by setting the `cache_seed` parameter.
+By default DiskCache uses `.cache` for storage. To change the cache directory,
+set `cache_path_root`:
+
+```python
+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:
+ client.create(..., cache=cache)
+```
### Turnning off cache
@@ -208,9 +226,16 @@ client = OpenAIWrapper(..., cache_seed=None)
client.create(..., cache_seed=None)
```
-_NOTE_. openai v1.1 introduces a new param `seed`. The difference between autogen's `cache_seed` and openai's `seed` is that:
-* autogen uses local disk cache to guarantee the exactly same output is produced for the same input and when cache is hit, no openai api call will be made.
-* openai's `seed` is a best-effort deterministic sampling with no guarantee of determinism. When using openai's `seed` with `cache_seed` set to None, even for the same input, an openai api call will be made and there is no guarantee for getting exactly the same output.
+### Difference between `cache_seed` and openai's `seed` parameter
+
+openai v1.1 introduces a new param `seed`.
+The differences between autogen's `cache_seed` and openai's `seed`:
+ - autogen uses local disk cache to guarantee the exactly same output is produced
+ for the same input and when cache is hit, no openai api call will be made.
+ - openai's `seed` is a best-effort deterministic sampling with no guarantee
+ of determinism. When using openai's `seed` with `cache_seed` set to None,
+ even for the same input, an openai api call will be made and there is
+ no guarantee for getting exactly the same output.
## Error handling