Skip to content

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

15.7 KB
Raw Blame

Caching

marimo comes with utilities to cache intermediate computations. These utilities can be applied as decorators to functions to cache their returned values; you can choose between saving caches in memory or to disk.

Basic usage

marimo provides two decorators for caching the return values of expensive functions:

  1. [mo.cache][marimo.cache], which saves cached values to memory;
  2. [mo.persistent_cache][marimo.persistent_cache], which saves cached values to disk.

/// tab | mo.cache

import marimo as mo

@mo.cache
def compute_embedding(data: str, embedding_dimension: int, model: str) -> np.ndarray:
    ...

///

/// tab | mo.persistent_cache

import marimo as mo

@mo.persistent_cache
def compute_embedding(data: str, embedding_dimension: int, model: str) -> np.ndarray:
    ...

///

/// tab | mo.cache (async)

import marimo as mo

@mo.cache
async def fetch_data(url: str, params: dict) -> dict:
    response = await http_client.get(url, params=params)
    return response.json()

///

/// tab | mo.persistent_cache (async)

import marimo as mo

@mo.persistent_cache
async def compute_embedding(data: str, embedding_dimension: int, model: str) -> np.ndarray:
    response = await llm_client.get_embeddings(data, model)
    return response.embeddings

///

Roughly speaking, the first time a cached function is called with a particular sequence of arguments, the function will run and its return value will be cached. The next time it is called with the same sequence of arguments (on cache hit), the function body will be skipped and the return value will be retrieved from cache instead.

The in-memory cache ([mo.cache][marimo.cache]) is faster and doesn't consume disk space, but it is lost on notebook restart. The disk cache ([mo.persistent_cache][marimo.persistent_cache]) is slower and consumes space on disk, but it persists across notebook runs, letting you pick up where you left off.

(For an in-memory cache of bounded size, use [mo.lru_cache][marimo.lru_cache].)

!!! note "Async functions are fully supported" All cache decorators (mo.cache, mo.lru_cache, mo.persistent_cache) work seamlessly with both synchronous and asynchronous functions. When multiple concurrent calls are made to a cached async function with the same arguments, only one execution occurs—the rest await the result. This prevents race conditions and duplicate work.

!!! tip "Where persistent caches are stored" By default, persistent caches are stored in __marimo__/cache/, in the directory of the current notebook. For projects versioned with git, consider adding **/__marimo__/cache/ to your .gitignore.

!!! tip "Caches are preserved even when a cell is re-run" If a cell defining a cached function is re-run, the cache will be preserved unless its source code (or the source code of the cell's ancestors) has changed.

Persistent cache context manager

You can also use marimo's [mo.persistent_cache][marimo.persistent_cache] as a context manager:

with mo.persistent_cache("my_cache_name"):
    X = my_expensive_computation(data, model)

The next time this block of code is run, if marimo detects a cache hit, the code will be skipped and your variables will be loaded into memory. The cache key for the context manager is computed in the same way as it is computed for decorated functions.

Automatic cell caching

mo.cache and mo.persistent_cache are opt-in per function or code block. marimo also has a notebook-wide mechanism that automatically attempts to cache every executed cell, not just the ones you have explicitly decorated.

Enable it with:

[tool.marimo.runtime]
cache_cells = true

!!! tip "Configuring for your project" Set the above snippet in your notebook's pyproject.toml as shown above, or directly in your notebooks' PEP 723 metadata.

With cache_cells enabled, marimo attempts to save and restore every cell's state on notebook restart. On a cache hit, marimo skips re-running the cell and hydrates its variables from a stored stub instead. If a stub cannot be hydrated (for example, its value is not hashable or serializable), marimo invalidates the cache for that cell and its producing ancestors, then re-runs them live. You do not need to clear anything manually.

!!! warning "Experimental" UI elements defined by a cached cell are not currently restored from cache on a cache hit, which forces a live re-run of that cell. Smarter caching (skipping cells where caching itself is slow) and surfacing more information about cache hits and misses in the UI are planned but not yet implemented.

Cached cell execution is also what powers cached WASM exports, which let you publish notebooks whose expensive or browser-incompatible cells were computed elsewhere beforehand.

!!! tip "Further reading" For more on the design of marimo's caching, see our SciPy paper (currently under peer review): github.com/scipy-conference/scipy_proceedings/pull/1262

Cache key

Both mo.cache and mo.persistent_cache use the same mechanism for creating a cache key, differing only in where the cache is stored. The cache key is based on function arguments and closed-over variables.

Function arguments

Arguments must either be primitive, marimo UI elements, array-like, or pickleable:

  1. Primitive types (strings, bytes, numbers, None) are hashed.
  2. marimo UI elements are hashed based on their value.
  3. Array-like objects are introspected, with their values being hashed.
  4. All other objects are pickled.

Closed-over variables

Syntactically closing over variables provides another way to parametrize functions. In this example, the variable x is "closed over":

x = 0
def my_function():
    return x + 1

Closed-over variables are processed in the following way:

  • marimo first attempts to hash or pickle the closed-over variables, just as it does for arguments.
  • If a closed-over variables cannot be hashed or pickled, then marimo uses the source code that defines the value as part of the cache key; in particular, marimo hashes the cell that defines the variable as well as the source code of that cell's ancestors. This assumes that the variable's value is a deterministic function of the source code that defines it, although certain side-effects (specifically, if a cell raised an exception or loaded from another cache) are taken into account.

Because marimo's cache key construction can fall back to source code for closed-over variables, closing over variables lets you cache functions even in the presence of non-hashable and non-pickleable arguments.

Limitations

marimo's cache has some limitations:

  • Side effects are not cached. This means that on a cache hit, side effects like printing, file I/O, or network requests will not occur.
  • The source code of imported modules is not used when computing the cache key.
    • By setting pin_modules to True, you can ensure that the cache is invalidated when module versions change (e.g., update when the module's __version__ attribute changes).
    • This limitation does not apply if the external module is a marimo notebook.
  • The return values of persistently cached functions must be serializable with pickle.

!!! warning "Don't mutate variables" marimo works best when you don't mutate variables across cells. The same is true for caching, since the cache key may not always be able to take mutations into account.

Decorators defined in other Python modules

Decorators defined in other Python modules that do not use functools.wraps cannot be correctly cached. This can lead to confusing bugs like the example below:

# my_lib.py
def my_decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper
# Cell 1
from my_lib import my_decorator

@mo.cache
@my_decorator
def expensive_function():
    # ... some computation
    return "result1"

@mo.cache
@my_decorator
def another_expensive_function():
    # ... different computation
    return "result2"

# This assertion may unexpectedly pass due to cache collision!
assert expensive_function() == another_expensive_function(), "But why?"

The fix is to make sure the decorator uses functools.wraps:

# my_lib.py (fixed)
from functools import wraps

def my_decorator(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

In this instance, the cache will work as expected because the decorated function has the same signature and metadata as the original function.

Cache signing and trust

By default, marimo uses the pickle module to serialize and deserialize cache entries. This is fast and safe for your local computer, but if you configure a shared cache directory or remote store, this may become a security issue since unpickling a cache entry is equivalent to running arbitrary code.

marimo has a cache verification mechanism built-in (which is currently opt-in). It uses digital signatures to verify that a cache entry was written by a trusted key and requires the cryptography package. That mechanism is the lazy loader: the storage backend selected with loader = "lazy" (the default) or method="lazy" on an individual cache. It is the only loader that signs and verifies. You can configure the verification level in your user configuration file at ~/.config/marimo/marimo.toml as follows:

[signing]
# Optional: This machine's signing identity.
# This will be generated if a key is not provided.
# MARIMO_CACHE_SIGNING_PRIVATE_KEY also sets this value.
private_key_path = "~/.marimo/cache_key.pem"

[signing.trusted_signers]
# Optional: If you use multiple caches, you can record trusted keys here.
# fingerprint = label. The label is a note to yourself
# Only the fingerprint matters.
"SHA256:kV9x2c...q8" = "CI cache key"
"SHA256:abc9f1...4d" = "Alice"

[cache]
verification = "on"  # "off" | "on" | "strict"
# Default `method` for `mo.persistent_cache`: "lazy" (default), "pickle", or "json".
# Only "lazy" can sign and verify.
loader = "lazy"

To sign and verify without a configuration file, set MARIMO_CACHE_SIGNING_PRIVATE_KEY to your signing key. To trust another key for verification only, set MARIMO_CACHE_SIGNING_PUBLIC_KEY to that key. Set each variable to the key itself in Privacy-Enhanced Mail (PEM) form, not a path.

!!! danger "Trusting a key grants code execution" Restoring a cache entry deserializes it with pickle, so trusting a fingerprint lets whoever holds that key run arbitrary code on your machine. There is no narrower "cache-only" version of this grant: add a fingerprint only if you would run any script that person handed you.

Cache entry off (default) on strict
Signed by a trusted key load load load
Unsigned, or signed by an unknown key load without verifying recompute raise an error

We recommend using strict in CI.

Comparison with functools.cache

Here is a table comparing marimo's cache with functools.cache:

Feature marimo cache functools.cache
Cache return values in memory? ✅ ✅
Cache return values to disk? ✅ ❌
Preserved on cell re-runs? ✅ ❌
Tracks closed-over variables ✅ ❌
Allows unhashable arguments? ✅ ❌
Allows Array-like arguments? ✅ ❌
Supports async functions? ✅ ❌
Suitable for lightweight functions (microseconds)? ❌ ✅

!!! tip "When to use functools.cache" Prefer functools.cache for extremely lightweight functions (that execute in less than a millisecond). Using memoization to calculate the Fibonacci sequence is a classic example of using functools.cache effectively. On a basic macbook in pure python, fib(35) takes 1 second to compute; with mo.cache it takes 0.000229 seconds; with functools.cache, it takes 0.000025 seconds (x9 faster!!). Although relatively small, the additional overhead of mo.cache (and more so mo.persistent_cache) is larger than functools.cache. If your function takes more than a few milliseconds to compute, the difference is negligible.

Tips

Isolate cached code blocks to their own cells

Isolating cached functions in separate cells improves cache reliability. When dependencies and cached functions are in the same cell, any change to the cell invalidates the cache, even if the cached function itself hasn't changed. Separating them ensures the cache is only invalidated when the function actually changes.

Don't do this:

# Cell 1
llm_client = ...
@mo.cache
def prompt_llm(query, **kwargs):
    message = {"role": "user", "content": query}
    return llm_client.chat.completions.create(messages=[message], **kwargs)

Do this instead:

# Cell 1
llm_client = ...
# Cell 2
@mo.cache
def prompt_llm(query, **kwargs):
    message = {"role": "user", "content": query}
    return llm_client.chat.completions.create(messages=[message], **kwargs)

Close over unhashable or un-pickleable arguments

The cache key is constructed in part by hashing or pickle-ing function arguments. When you call a cached function with arguments that cannot be processed in this way, an exception will be raised. To parametrize cached functions with unhashable or un-pickleable arguments, syntactically close over them instead.

You can't do this:

# Cell 1
@mo.cache
def query_database(query, engine):
    return engine.execute(query)
# This won't work because my_database_engine is not hashable
query_database("SELECT * FROM my_table", my_database_engine)

Instead, you can close over my_database_engine:

Do this:

# Cell 1
my_database_engine = ...
# Cell 2
@mo.cache
def query_database(query):
    return my_database_engine.execute(query)

Close-over low-memory-footprint variables

Non-primitive closed-over variables are serialized for cache key generation. When possible, compute derived values (like length) outside the cache block and only use the small values inside.

Don't do this:

with mo.persistent_cache("bad example"):
    length = len(my_very_large_dataset)
    ... # uses length

Do this instead:

length = len(my_very_large_dataset)  # my_very_large_dataset is not needed for cache invalidation
with mo.persistent_cache("good example"):
    ... # uses length

Use mo.watch.file when working with files

Don't do this:

my_file = open("my_file.txt")
with mo.persistent_cache("my_file"):
    data = my_file.read()
    # Do something with data
my_file.close()

Do this instead:

# Cell 1
my_file = mo.watch.file("my_file.txt")
# Cell 2
with mo.persistent_cache("my_file"):
    data = my_file.read()
    # Do something with data

API

::: marimo.cache ::: marimo.lru_cache ::: marimo.persistent_cache