Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Add persistent LocalInterpreter (#10238)

Isaac Millerd58c3007ebd3
diff --git a/docs/docs/api/modules/Flex.md b/docs/docs/api/modules/Flex.md
index 826ff5045..0123341c9 100644
--- a/docs/docs/api/modules/Flex.md
+++ b/docs/docs/api/modules/Flex.md
@@ -27,7 +27,7 @@ print(result.total_cents)
Out of the box, `solve` is just a `dspy.Predict` over the signature, wrapped in a module (with `tools`, it starts as a `dspy.RLM` instead — see [Tools](#tools)). The point of `Flex` is what happens when you optimize it (see [Optimizing with GEPA](#optimizing-with-gepa)): GEPA can replace that baseline with, say, a predictor that only extracts quantities and unit prices, and a line of Python that multiplies and sums them.
-The generated code always runs in a sandbox (`interpreter_factory` defaults to `dspy.PythonInterpreter`), so the example above needs [Deno](https://deno.land/) installed — see [Sandboxed Execution](#sandboxed-execution).
+The generated code always runs through a `CodeInterpreter` (`interpreter_factory` defaults to the sandboxed `dspy.PythonInterpreter`), so the example above needs [Deno](https://deno.land/) installed — see [Interpreter Execution](#interpreter-execution).
## How Optimization Works
@@ -82,9 +82,9 @@ def metric(gold, pred, trace=None, pred_name=None, pred_trace=None, program_trac
The `program_trace` parameter is opt-in *by declaration*: only metrics that name it receive the trace. Keep the penalty small relative to correctness, so a decomposition has to *hold* accuracy to win.
-## Sandboxed Execution
+## Interpreter Execution
-`Flex` always runs its generated code in a sandbox — never in the host Python process. `interpreter_factory` defaults to `dspy.PythonInterpreter` (Deno/Pyodide) and must be a **zero-argument factory** returning a fresh `CodeInterpreter`; a bare instance is not accepted, so parallel evaluations receive isolated sessions. The factory is called once per sandbox session, including separate sessions requested by nested code-executing modules. The code is authored by the reflection model, so isolating it keeps it from running with your host's full permissions. With the default interpreter, optimizer-authored control flow, string work, arithmetic, and supported imports run inside the sandbox, and only provided-tool calls, predictor construction, and predictor calls bridge back to the host, which makes the real LM calls.
+`Flex` always runs its generated code through a `CodeInterpreter`. `interpreter_factory` defaults to `dspy.PythonInterpreter` (Deno/Pyodide) and must be a **zero-argument factory** returning a fresh interpreter; a bare instance is not accepted, so parallel evaluations receive isolated sessions. The factory is called once per interpreter session, including separate sessions requested by nested code-executing modules. With the default interpreter, optimizer-authored control flow, string work, arithmetic, and supported imports run inside the sandbox, and only provided-tool calls, predictor construction, and predictor calls bridge back to the host, which makes the real LM calls. Custom factories define their own trust boundary: `dspy.LocalInterpreter`, for example, separates process memory and stdout but retains the host user's filesystem, environment, credentials, network, and process authority.
Because the default builds a `PythonInterpreter`, *running* a `Flex` needs [Deno](https://deno.land/) installed; without it, the call raises.
@@ -130,7 +130,7 @@ The interpreter is a **runtime dependency and is not serialized**. Reconstructin
|-----------|------|---------|-------------|
| `signature` | `str \| Signature` | required | Declares the module's inputs and outputs (e.g. `"invoice -> total_cents: int"`). |
| `tools` | `list[Callable \| dspy.Tool]` | `None` | Tools the generated code may call. With tools, the baseline is a `dspy.RLM`; without, a `dspy.Predict`. |
-| `interpreter_factory` | `Callable[[], CodeInterpreter]` | `PythonInterpreter` | Zero-arg factory returning a fresh `CodeInterpreter` for each sandbox session; defaults to `dspy.PythonInterpreter` (needs Deno). A bare interpreter instance is not accepted. Supported Python and libraries are interpreter-dependent. |
+| `interpreter_factory` | `Callable[[], CodeInterpreter]` | `PythonInterpreter` | Zero-arg factory returning a fresh `CodeInterpreter` for each interpreter session; defaults to `dspy.PythonInterpreter` (needs Deno). A bare interpreter instance is not accepted. Supported Python, libraries, and security boundaries are interpreter-dependent. |
| `max_predictor_calls` | `int \| None` | `100` | Maximum number of predictor calls the generated code can make in one `forward` — a guard against runaway loops. `None` removes the limit. |
## Notes
@@ -139,7 +139,7 @@ The interpreter is a **runtime dependency and is not serialized**. Reconstructin
`Flex` is marked experimental. The API and the optimization behavior may change between releases; pin a version if you depend on it.
!!! note "Interpreter Requirements"
- `Flex` always runs generated code in a sandbox (`interpreter_factory` defaults to `dspy.PythonInterpreter`), which requires [Deno](https://deno.land/) for its Pyodide WASM sandbox — see the [RLM page](RLM.md#deno-installation) for installation notes.
+ `Flex` defaults to `dspy.PythonInterpreter`, which requires [Deno](https://deno.land/) for its Pyodide WASM sandbox — see the [RLM page](RLM.md#deno-installation) for installation notes. `dspy.LocalInterpreter` requires no additional runtime but is not a security sandbox.
## API Reference
diff --git a/docs/docs/api/modules/RLM.md b/docs/docs/api/modules/RLM.md
index 1998e9864..bc59bf8ac 100644
--- a/docs/docs/api/modules/RLM.md
+++ b/docs/docs/api/modules/RLM.md
@@ -270,6 +270,9 @@ RLM returns a `Prediction` with:
!!! note "Interpreter Requirements"
The default `PythonInterpreter` requires [Deno](https://deno.land/) to be installed for the Pyodide WASM sandbox.
+ `LocalInterpreter` requires no additional runtime and provides a separate CPython process, but it is not a
+ security sandbox: generated code retains the host user's files, environment, credentials, network, and process
+ authority.
## API Reference
diff --git a/docs/docs/api/tools/LocalInterpreter.md b/docs/docs/api/tools/LocalInterpreter.md
new file mode 100644
index 000000000..0b5201ea9
--- /dev/null
+++ b/docs/docs/api/tools/LocalInterpreter.md
@@ -0,0 +1,53 @@
+# dspy.LocalInterpreter
+
+`LocalInterpreter` runs Python in one persistent local CPython worker. It is useful when generated code needs
+ordinary Python compatibility plus separation from the DSPy process's memory, stdout, and lifecycle. State and
+imports persist until `shutdown()`, host tools cross a JSON protocol, and `execution_timeout` can terminate a stuck
+worker.
+
+```python
+import dspy
+
+rlm = dspy.RLM(
+ "question: str -> answer: int",
+ interpreter_factory=dspy.LocalInterpreter,
+)
+```
+
+!!! warning "A subprocess is not a security sandbox"
+ Generated code retains the host user's filesystem, environment, credentials, subprocess, and network authority.
+ Use the default [`PythonInterpreter`](PythonInterpreter.md) or a remote sandbox for untrusted code. The subprocess
+ boundary protects ordinary host memory and stdout from accidental mutation; it does not contain hostile code.
+
+Inputs, host-tool arguments/results, and structured outputs must be JSON-compatible. The worker uses the current
+Python executable.
+`execution_timeout` includes time spent in host tools and terminates the worker promptly when the deadline expires.
+Python cannot forcibly stop a running host callable, so that callable may finish in a detached daemon thread; its
+result is discarded and the interpreter session remains terminal.
+
+Parallel guest code is supported when all threads finish before the current `execute()` call returns, such as a
+context-managed `ThreadPoolExecutor`. Leaving a guest thread running makes the session terminal because that thread
+could otherwise mutate state, write output, or invoke tools during a later execution. Shutdown and terminal failures
+also terminate ordinary descendant processes with the worker.
+
+`LocalInterpreter.execution_instructions` is stable class metadata. `RLM` adds it to the action prompt without
+starting a worker.
+
+<!-- START_API_REF -->
+::: dspy.LocalInterpreter
+ handler: python
+ options:
+ members:
+ - __call__
+ - execute
+ - shutdown
+ - start
+ show_source: true
+ show_root_heading: true
+ heading_level: 2
+ docstring_style: google
+ show_root_full_path: true
+ show_object_full_path: false
+ separate_signature: false
+ inherited_members: true
+<!-- END_API_REF -->
diff --git a/docs/docs/api/tools/index.md b/docs/docs/api/tools/index.md
index 91c829e96..30668c38b 100644
--- a/docs/docs/api/tools/index.md
+++ b/docs/docs/api/tools/index.md
@@ -5,5 +5,6 @@ API reference for DSPy tools. Select a page below.
<!-- START_API_INDEX -->
- [ColBERTv2](ColBERTv2.md)
- [Embeddings](Embeddings.md)
+- [LocalInterpreter](LocalInterpreter.md)
- [PythonInterpreter](PythonInterpreter.md)
<!-- END_API_INDEX -->
diff --git a/docs/docs/diving-deeper/flex.md b/docs/docs/diving-deeper/flex.md
index 5b7059019..b1116893f 100644
--- a/docs/docs/diving-deeper/flex.md
+++ b/docs/docs/diving-deeper/flex.md
@@ -32,9 +32,9 @@ When you mix a `Flex` with ordinary modules in one program, GEPA optimizes the `
The reflection model authors code, and code can be wrong. A candidate that doesn't parse raises when GEPA binds it to build the program for evaluation; `Flex` optimization catches that, logs it, and scores the whole batch at the failure score, letting the search continue and simply not select the broken candidate. Code that parses but breaks when it runs — an import that doesn't resolve, an edge case that throws on some inputs — fails per example at `forward` instead: each crashed example is scored at the failure score in its own slot, by example index, so the surviving scores stay aligned to the batch and GEPA's per-instance bookkeeping stays intact. Either way, the optimization is robust to the reflection model's mistakes by construction.
-### 7. Generated code always runs in an interpreter, never in-process
+### 7. Generated code always runs through an interpreter
-`Flex` runs `module_src` in a sandbox: `interpreter_factory` defaults to `dspy.PythonInterpreter` (Deno/Pyodide) and must be a *zero-argument factory* — a bare instance or `None` is rejected. Since the code is authored by the reflection model, it never gets the host's full permissions: everything it does stays in the sandbox except provided-tool calls, predictor construction, and predictor calls, which bridge back to the host. The factory creates a fresh interpreter for each sandbox session; a forward owns an outer session and nested code-executing modules may request separate sessions. Source portability between custom interpreters is not guaranteed. `max_predictor_calls` caps how many predictor calls the generated code can make in one `forward`.
+`interpreter_factory` defaults to `dspy.PythonInterpreter` (Deno/Pyodide) and must be a *zero-argument factory* — a bare instance or `None` is rejected. With the default, everything stays in the sandbox except provided-tool calls, predictor construction, and predictor calls, which bridge back to the host. Custom factories define their own trust boundary: `LocalInterpreter` separates process memory and stdout but retains the host user's filesystem, environment, credentials, network, and process authority. The factory creates a fresh interpreter for each session; a forward owns an outer session and nested code-executing modules may request separate sessions. Source portability between custom interpreters is not guaranteed. `max_predictor_calls` caps how many predictor calls the generated code can make in one `forward`.
### 8. The declared output types are enforced at the sandbox boundary
@@ -78,7 +78,7 @@ Add `program_trace=None` as a sixth parameter to your metric and GEPA passes the
Plain functions or `dspy.Tool` instances, referenced by name in the generated code, so each name must be a valid Python identifier. Providing tools makes the baseline a `dspy.RLM` and tells the code proposer they are in scope — it can wire them into `dspy.RLM`/`dspy.ReAct`, call them directly, or supplement them with its own inline helpers.
**`interpreter_factory=...`**
-Defaults to `dspy.PythonInterpreter` (sandboxed, needs Deno). Must be a zero-argument callable returning a fresh `CodeInterpreter` for each sandbox session; parallel evaluations and nested code-executing modules can therefore receive isolated sessions. As in `dspy.RLM`, a bare interpreter instance is not accepted. This low-level hook does not guarantee source or standard-library portability between different interpreters.
+Defaults to `dspy.PythonInterpreter` (sandboxed, needs Deno). Must be a zero-argument callable returning a fresh `CodeInterpreter` for each interpreter session; parallel evaluations and nested code-executing modules can therefore receive isolated sessions. As in `dspy.RLM`, a bare interpreter instance is not accepted. This low-level hook does not guarantee source, standard-library portability, or a particular security boundary between different interpreters.
**`max_predictor_calls`**
The maximum number of predictor calls the generated code can make in one `forward`. It guards against runaway loops. `None` removes the limit.
diff --git a/docs/docs/diving-deeper/rlm.md b/docs/docs/diving-deeper/rlm.md
index ceb0d96a6..09fb1f928 100644
--- a/docs/docs/diving-deeper/rlm.md
+++ b/docs/docs/diving-deeper/rlm.md
@@ -30,9 +30,9 @@ Recursion is the expensive part, so `dspy.RLM` bounds it. `max_llm_calls` caps h
The model steering the loop and the model answering snippets need not be the same. `sub_lm` sets the model for `llm_query`, falling back to `dspy.settings.lm` when unset. A common split pairs a strong model for planning with a cheap model for extraction, since reading one snippet is simpler than orchestrating the whole search. You pay for the strong model only on the calls that need judgment.
-### 7. Generated code runs in a pluggable sandboxed interpreter
+### 7. Generated code runs in a pluggable interpreter
-The model’s code is untrusted, so it runs in a sandbox. The default `PythonInterpreter` executes Python in a Deno and Pyodide WASM runtime with no filesystem or network access. Each `forward()` creates one interpreter, uses it for the whole REPL loop, and shuts it down afterward. Supply `interpreter_factory=` to configure another `CodeInterpreter`, such as an adapter for a remote sandbox.
+The default `PythonInterpreter` executes Python in a Deno and Pyodide WASM sandbox with no filesystem or network access. Each `forward()` creates one interpreter, uses it for the whole REPL loop, and shuts it down afterward. Supply `interpreter_factory=` to configure another `CodeInterpreter`. `LocalInterpreter` offers ordinary CPython in a separate local process, but it is for trusted code because it retains the host user's filesystem, environment, credentials, network, and process authority.
### 8. `SandboxSerializable` loads large inputs into the sandbox once
@@ -90,7 +90,7 @@ The model for `llm_query` and `llm_query_batched`. Left unset it falls back to `
A list of plain functions or `dspy.Tool` objects. RLM normalizes each to a `Tool`, rejects names that aren’t valid identifiers or that collide with the built-ins, and documents their signatures in the action prompt. The model calls them as ordinary Python inside its code.
**`interpreter_factory=...`**
-A zero-argument callable that returns a fresh `CodeInterpreter` for one invocation. RLM may call the factory concurrently, and it always shuts down the returned interpreter. A class such as `PythonInterpreter` is already a factory; use `functools.partial` or a callable provider object when construction needs configuration. RLM adds invocation-scoped tools to the returned interpreter's mutable `tools` dictionary, so remote sandboxes need a `CodeInterpreter` adapter that supports that protocol.
+A zero-argument callable that returns a fresh `CodeInterpreter` for one invocation. RLM may call the factory concurrently, and it always shuts down the returned interpreter. A class such as `PythonInterpreter` or `LocalInterpreter` is already a factory; use `functools.partial` or a callable provider object when construction needs configuration. RLM adds invocation-scoped tools to the returned interpreter's mutable `tools` dictionary, so remote sandboxes need a `CodeInterpreter` adapter that supports that protocol.
**`__call__(interpreter, **inputs)` / `acall(interpreter, **inputs)`**
An escape hatch for a caller-owned interpreter, supplied as the first positional argument. RLM mutates its `tools` dictionary and, when supported, its output-field metadata, but does not shut down or restore the instance. Reuse is supported only for sequential calls to the same RLM instance, so retained variables and tool registrations stay within one program and trust boundary. Use `interpreter_factory` for concurrent invocations. A `PythonInterpreter` override must also stay on the thread where it was first used.
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index fd59857b9..e1cd5cab9 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -178,6 +178,7 @@ nav:
- api/tools/index.md
- ColBERTv2: api/tools/ColBERTv2.md
- Embeddings: api/tools/Embeddings.md
+ - LocalInterpreter: api/tools/LocalInterpreter.md
- PythonInterpreter: api/tools/PythonInterpreter.md
- Utils:
- api/utils/index.md
diff --git a/dspy/predict/flex/flex.py b/dspy/predict/flex/flex.py
index 554ddb19f..ea3586fc3 100644
--- a/dspy/predict/flex/flex.py
+++ b/dspy/predict/flex/flex.py
@@ -37,7 +37,7 @@ class Flex(Module, Parameter):
signature: A ``dspy.Signature`` class or string declaring inputs/outputs.
tools: ``dspy.Tool`` instances or named callables.
interpreter_factory: Zero-argument callable returning a fresh ``CodeInterpreter`` for each
- sandbox session. Defaults to ``dspy.PythonInterpreter`` (sandbox, requires Deno).
+ interpreter session. Defaults to ``dspy.PythonInterpreter`` (sandbox, requires Deno).
max_predictor_calls: Maximum number of predictor calls the optimizer-authored code can
make per ``forward`. ``None`` removes the limit.
"""
diff --git a/dspy/primitives/__init__.py b/dspy/primitives/__init__.py
index f1ad6bf32..dc768cfa2 100644
--- a/dspy/primitives/__init__.py
+++ b/dspy/primitives/__init__.py
@@ -1,6 +1,7 @@
from dspy.primitives.base_module import BaseModule
from dspy.primitives.code_interpreter import CodeExecutionError, CodeInterpreter, CodeInterpreterError, FinalOutput
from dspy.primitives.example import Example
+from dspy.primitives.local_interpreter import LocalInterpreter
from dspy.primitives.module import Module
from dspy.primitives.prediction import Completions, Prediction
from dspy.primitives.python_interpreter import PythonInterpreter
@@ -14,6 +15,7 @@ __all__ = [
"Example",
"FinalOutput",
"CodeInterpreterError",
+ "LocalInterpreter",
"Module",
"Prediction",
"PythonInterpreter",
diff --git a/dspy/primitives/code_interpreter.py b/dspy/primitives/code_interpreter.py
index 8a0393ebf..a5f232227 100644
--- a/dspy/primitives/code_interpreter.py
+++ b/dspy/primitives/code_interpreter.py
@@ -4,6 +4,7 @@ Abstract interpreter interface for code execution environments.
This module defines the CodeInterpreter protocol that allows RLM and other
code-executing modules to work with different interpreter implementations:
- PythonInterpreter: Local Deno/Pyodide WASM interpreter
+- LocalInterpreter: Persistent local CPython worker
- MockInterpreter: Scriptable responses for testing
"""
@@ -67,7 +68,8 @@ class CodeInterpreter(Protocol):
4. shutdown() - Release resources
Example implementations:
- - LocalInterpreter: Deno/Pyodide WASM interpreter (local)
+ - PythonInterpreter: Deno/Pyodide WASM interpreter (local sandbox)
+ - LocalInterpreter: Persistent local CPython worker (not a sandbox)
- MockInterpreter: Scriptable responses for testing
Pooling:
diff --git a/dspy/primitives/local_interpreter.py b/dspy/primitives/local_interpreter.py
new file mode 100644
index 000000000..0793cd563
--- /dev/null
+++ b/dspy/primitives/local_interpreter.py
@@ -0,0 +1,319 @@
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import contextvars
+import inspect
+import json
+import keyword
+import os
+import queue
+import signal
+import subprocess
+import sys
+import threading
+import time
+from collections.abc import Callable
+from concurrent.futures import Future
+from concurrent.futures import TimeoutError as FutureTimeoutError
+from pathlib import Path
+from typing import Any, NoReturn
+
+from dspy.primitives.code_interpreter import CodeExecutionError, CodeInterpreterError, FinalOutput
+from dspy.utils.callback import BaseCallback, with_callbacks
+
+
+def _run_in_thread(function: Callable[[], Any]) -> Future[Any]:
+ outcome = Future()
+
+ def run() -> None:
+ try:
+ outcome.set_result(function())
+ except BaseException as exc:
+ outcome.set_exception(exc)
+
+ threading.Thread(target=contextvars.copy_context().run, args=(run,), daemon=True).start()
+ return outcome
+
+
+def _await_in_sync(awaitable: Any) -> Any:
+ try:
+ asyncio.get_running_loop()
+ except RuntimeError:
+ return asyncio.run(awaitable)
+ return _run_in_thread(lambda: asyncio.run(awaitable)).result()
+
+
+class LocalInterpreter:
+ """Execute Python in a persistent local subprocess.
+
+ This separates generated code from DSPy's memory, stdout, and lifecycle,
+ but it is not a security sandbox. The worker retains the host user's files,
+ environment, credentials, subprocess, and network authority.
+
+ Args:
+ tools: Host functions exposed to executed code by name. Arguments and
+ return values must be JSON-compatible.
+ output_fields: Optional field definitions for typed ``SUBMIT`` calls.
+ execution_timeout: Maximum seconds for one execution, including host
+ tool calls. A timeout terminates the worker and its session state.
+ Python cannot forcibly stop a running host callable, so a timed-out
+ callable may finish in a detached daemon thread; its result is discarded.
+ callbacks: Optional instance-level callback handlers.
+ """
+
+ execution_instructions = (
+ "Code runs as Python in a persistent local subprocess. State, imports, functions, and JSON-compatible "
+ "variables persist for this session. Host tools and SUBMIT are available as global functions. All threads "
+ "must finish before each code execution returns; use context-managed executors for parallel work."
+ )
+
+ def __init__(
+ self,
+ tools: dict[str, Callable[..., Any]] | None = None,
+ output_fields: list[dict[str, Any]] | None = None,
+ *,
+ execution_timeout: float | None = None,
+ callbacks: list[BaseCallback] | None = None,
+ ) -> None:
+ if execution_timeout is not None and execution_timeout <= 0:
+ raise ValueError("execution_timeout must be positive or None")
+ self._validate_tool_names(tools or {})
+ self.tools = dict(tools or {})
+ fields = None if output_fields is None else [dict(field) for field in output_fields]
+ self._validate_output_fields(fields)
+ self.output_fields = fields
+ self.execution_timeout = execution_timeout
+ self.callbacks = list(callbacks or [])
+ self._process: subprocess.Popen[str] | None = None
+ self._responses: queue.Queue[str | None] = queue.Queue()
+ self._lock = threading.Lock()
+ self._send_lock = threading.Lock()
+ self._ended = False
+
+ @staticmethod
+ def _validate_tool_names(tools: dict[str, Callable[..., Any]]) -> None:
+ invalid = [
+ name
+ for name in tools
+ if not isinstance(name, str)
+ or not name.isidentifier()
+ or keyword.iskeyword(name)
+ or name in {"SUBMIT", "__builtins__"}
+ ]
+ if invalid:
+ raise ValueError(f"tool names must be non-reserved Python identifiers; invalid names: {invalid!r}")
+
+ @staticmethod
+ def _validate_output_fields(output_fields: list[dict[str, Any]] | None) -> None:
+ if output_fields is None:
+ return
+ names = [field.get("name") if isinstance(field, dict) else None for field in output_fields]
+ if any(not isinstance(name, str) or not name.isidentifier() or keyword.iskeyword(name) for name in names):
+ raise ValueError("output field names must be non-keyword Python identifiers")
+ if len(names) != len(set(names)):
+ raise ValueError("output field names must be unique")
+ try:
+ json.dumps(output_fields, allow_nan=False)
+ except (TypeError, ValueError) as exc:
+ raise ValueError(f"output fields must be JSON-compatible: {exc}") from exc
+
+ @with_callbacks
+ def start(self) -> None:
+ """Start the worker, or return immediately if it is already running."""
+ if self._ended:
+ raise CodeInterpreterError("LocalInterpreter session has been shut down.")
+ if self._process is not None:
+ return
+ try:
+ process = subprocess.Popen(
+ [sys.executable, "-I", str(Path(__file__).with_name("local_interpreter_worker.py"))],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.DEVNULL,
+ creationflags=subprocess.CREATE_NEW_PROCESS_GROUP if os.name == "nt" else 0,
+ start_new_session=os.name == "posix",
+ text=True,
+ )
+ except OSError as exc:
+ raise CodeInterpreterError(f"Unable to start Python worker: {exc}") from exc
+ self._process = process
+ threading.Thread(target=self._read_responses, args=(process,), daemon=True).start()
+ message = self._receive(time.monotonic() + 10, "Python worker did not start within 10 seconds.")
+ if message.get("type") != "ready":
+ self._raise_terminal_error(f"Python worker returned an invalid startup message: {message!r}")
+
+ def _read_responses(self, process: subprocess.Popen[str]) -> None:
+ assert process.stdout is not None
+ for line in process.stdout:
+ self._responses.put(line)
+ self._responses.put(None)
+
+ def _send(self, message: dict[str, Any], process: subprocess.Popen[str] | None = None) -> None:
+ with self._send_lock:
+ current = self._process
+ if process is not None and current is not process:
+ return
+ assert current is not None and current.stdin is not None
+ try:
+ print(json.dumps(message, separators=(",", ":"), allow_nan=False), file=current.stdin, flush=True)
+ except (OSError, TypeError, ValueError) as exc:
+ self._raise_terminal_error(f"Unable to send request to Python worker: {exc}", exc)
+
+ def _receive(self, deadline: float | None, timeout_message: str) -> dict[str, Any]:
+ timeout = None if deadline is None else max(0, deadline - time.monotonic())
+ try:
+ line = self._responses.get(timeout=timeout)
+ except queue.Empty:
+ self._raise_terminal_error(timeout_message)
+ if line is None:
+ self._raise_terminal_error("Python worker exited unexpectedly.")
+ try:
+ message = json.loads(line)
+ except json.JSONDecodeError as exc:
+ self._raise_terminal_error(f"Python worker returned invalid JSON: {line!r}", exc)
+ if not isinstance(message, dict) or not isinstance(message.get("type"), str):
+ self._raise_terminal_error(f"Python worker returned an invalid message: {message!r}")
+ return message
+
+ def _raise_terminal_error(self, message: str, cause: BaseException | None = None) -> NoReturn:
+ self._ended = True
+ self._kill()
+ raise CodeInterpreterError(message) from cause
+
+ def _kill(self) -> None:
+ process = self._process
+ self._process = None
+ if process is None:
+ return
+ if os.name == "posix":
+ with contextlib.suppress(ProcessLookupError, PermissionError):
+ os.killpg(process.pid, signal.SIGKILL)
+ elif os.name == "nt":
+ with contextlib.suppress(OSError):
+ subprocess.run(
+ ["taskkill", "/F", "/T", "/PID", str(process.pid)],
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ check=False,
+ )
+ with contextlib.suppress(OSError):
+ process.kill()
+ with contextlib.suppress(subprocess.TimeoutExpired):
+ process.wait(timeout=1)
+ for stream in (process.stdin, process.stdout):
+ if stream is not None:
+ stream.close()
+
+ @with_callbacks
+ def invoke_tool(self, tool_name: str, args: list[Any], kwargs: dict[str, Any]) -> Any:
+ """Invoke one currently registered host tool."""
+ if tool_name not in self.tools:
+ raise CodeInterpreterError(f"Unknown tool: {tool_name}")
+ result = self.tools[tool_name](*args, **kwargs)
+ return _await_in_sync(result) if inspect.isawaitable(result) else result
+
+ def _handle_tool(self, request: dict[str, Any], process: subprocess.Popen[str]) -> None:
+ try:
+ value = self.invoke_tool(request["name"], request.get("args", []), request.get("kwargs", {}))
+ json.dumps(value, allow_nan=False)
+ response = {"type": "tool_result", "id": request["id"], "value": value}
+ except Exception as exc:
+ response = {"type": "tool_error", "id": request["id"], "error": f"{type(exc).__name__}: {exc}"}
+ self._send(response, process)
+
+ @with_callbacks
+ def execute(self, code: str, variables: dict[str, Any] | None = None) -> Any:
+ """Execute code in the worker's persistent namespace."""
+ if not self._lock.acquire(blocking=False):
+ raise CodeInterpreterError("LocalInterpreter already has an active execution.")
+ try:
+ variables = {} if variables is None else variables
+ self._validate_tool_names(self.tools)
+ self._validate_output_fields(self.output_fields)
+ if not isinstance(code, str):
+ raise CodeInterpreterError("code must be a string")
+ if not isinstance(variables, dict):
+ raise CodeInterpreterError("variables must map Python identifiers to JSON-compatible values")
+ invalid_variables = [
+ name
+ for name in variables
+ if not isinstance(name, str)
+ or not name.isidentifier()
+ or keyword.iskeyword(name)
+ or name in {"SUBMIT", "__builtins__"}
+ or name in self.tools
+ ]
+ if invalid_variables:
+ raise CodeInterpreterError(
+ "variable names must be non-reserved Python identifiers distinct from tool names; "
+ f"invalid names: {invalid_variables!r}"
+ )
+ try:
+ json.dumps(variables, allow_nan=False)
+ except (TypeError, ValueError) as exc:
+ raise CodeInterpreterError(f"variables must be JSON-compatible: {exc}") from exc
+
+ if self._process is None:
+ self.start()
+ deadline = None if self.execution_timeout is None else time.monotonic() + self.execution_timeout
+ self._send(
+ {
+ "type": "execute",
+ "code": code,
+ "variables": variables,
+ "tools": list(self.tools),
+ "output_fields": self.output_fields,
+ }
+ )
+ timeout_message = (
+ "Python worker did not respond."
+ if self.execution_timeout is None
+ else f"Python worker exceeded execution timeout of {self.execution_timeout:g} seconds."
+ )
+ process = self._process
+ assert process is not None
+ tool_calls: list[Future[Any]] = []
+ message = self._receive(deadline, timeout_message)
+ while message.get("type") == "tool_request":
+ tool_calls.append(_run_in_thread(lambda request=message: self._handle_tool(request, process)))
+ message = self._receive(deadline, timeout_message)
+ for tool_call in tool_calls:
+ timeout = None if deadline is None else max(0, deadline - time.monotonic())
+ try:
+ tool_call.result(timeout)
+ except FutureTimeoutError:
+ self._raise_terminal_error(timeout_message)
+ kind = message.get("type")
+ if kind == "terminal_error":
+ self._raise_terminal_error(f"Python worker failed: {message.get('error')}")
+ if kind == "syntax":
+ raise SyntaxError(message.get("error"))
+ if kind == "execution_error":
+ raise CodeExecutionError(message.get("error"))
+ if kind == "final":
+ return FinalOutput(message.get("value"))
+ if kind == "result":
+ value, stdout = message.get("value"), message.get("stdout")
+ return value if value is not None else (stdout or None)
+ self._raise_terminal_error(f"Python worker returned an unknown message: {message!r}")
+ finally:
+ self._lock.release()
+
+ def __enter__(self) -> LocalInterpreter:
+ self.start()
+ return self
+
+ def __exit__(self, *_: Any) -> None:
+ self.shutdown()
+
+ def __call__(self, code: str, variables: dict[str, Any] | None = None) -> Any:
+ return self.execute(code, variables)
+
+ @with_callbacks
+ def shutdown(self) -> None:
+ """Terminate the worker and discard its session state."""
+ if self._ended:
+ return
+ self._ended = True
+ self._kill()
diff --git a/dspy/primitives/local_interpreter_worker.py b/dspy/primitives/local_interpreter_worker.py
new file mode 100644
index 000000000..50522aacb
--- /dev/null
+++ b/dspy/primitives/local_interpreter_worker.py
@@ -0,0 +1,236 @@
+from __future__ import annotations
+
+import ast
+import builtins
+import contextlib
+import itertools
+import json
+import keyword
+import os
+import queue
+import sys
+import tempfile
+import threading
+from typing import Any
+
+_protocol_output = os.fdopen(os.dup(sys.__stdout__.fileno()), "w", encoding="utf-8")
+os.set_inheritable(_protocol_output.fileno(), False)
+_worker_stdout, _worker_stderr = sys.stdout, sys.stderr
+_sink = os.open(os.devnull, os.O_WRONLY)
+os.dup2(_sink, 1)
+os.dup2(_sink, 2)
+os.close(_sink)
+_send_lock = threading.Lock()
+
+
+class Submission(BaseException):
+ def __init__(self, value: Any) -> None:
+ self.value = value
+
+
+class Protocol:
+ def __init__(self) -> None:
+ self.commands: queue.Queue[dict[str, Any] | None] = queue.Queue()
+ self.pending: dict[int, queue.Queue[dict[str, Any] | None]] = {}
+ self.lock = threading.Lock()
+ self.ids = itertools.count()
+ threading.Thread(target=self._read, daemon=True).start()
+
+ def _read(self) -> None:
+ try:
+ while True:
+ message = receive()
+ request_id = message.get("id")
+ if message.get("type") in {"tool_result", "tool_error"} and isinstance(request_id, int):
+ with self.lock:
+ response = self.pending.get(request_id)
+ if response is None:
+ raise ValueError("response does not match a pending tool request")
+ response.put(message)
+ else:
+ self.commands.put(message)
+ except EOFError:
+ self.commands.put(None)
+ with self.lock:
+ pending = list(self.pending.values())
+ for response in pending:
+ response.put(None)
+ except BaseException as exc:
+ self.commands.put({"type": "protocol_error", "error": describe(exc)})
+
+ def call_tool(self, name: str, args: tuple[Any, ...], kwargs: dict[str, Any]) -> Any:
+ request_id = next(self.ids)
+ response: queue.Queue[dict[str, Any] | None] = queue.Queue(maxsize=1)
+ with self.lock:
+ self.pending[request_id] = response
+ try:
+ send({"type": "tool_request", "id": request_id, "name": name, "args": args, "kwargs": kwargs})
+ message = response.get()
+ finally:
+ with self.lock:
+ self.pending.pop(request_id, None)
+ if message is None:
+ raise EOFError
+ if message.get("type") == "tool_error":
+ raise RuntimeError(message.get("error"))
+ return message.get("value")
+
+
+class CapturedOutput:
+ def __enter__(self) -> CapturedOutput:
+ self.file = tempfile.TemporaryFile()
+ sys.stdout, sys.stderr = _worker_stdout, _worker_stderr
+ for stream in (_worker_stdout, _worker_stderr):
+ with contextlib.suppress(Exception):
+ stream.flush()
+ self.saved = os.dup(1), os.dup(2)
+ os.dup2(self.file.fileno(), 1)
+ os.dup2(self.file.fileno(), 2)
+ return self
+
+ def __exit__(self, *_: Any) -> None:
+ for stream in (sys.stdout, sys.stderr):
+ with contextlib.suppress(Exception):
+ stream.flush()
+ sys.stdout, sys.stderr = _worker_stdout, _worker_stderr
+ os.dup2(self.saved[0], 1)
+ os.dup2(self.saved[1], 2)
+ os.close(self.saved[0])
+ os.close(self.saved[1])
+ self.file.seek(0)
+ self.value = self.file.read().decode(errors="replace").rstrip("\n")
+ self.file.close()
+
+
+class Session:
+ def __init__(self, protocol: Protocol) -> None:
+ self.protocol = protocol
+ self.worker_threads = set(threading.enumerate())
+ self.namespace: dict[str, Any] = {"__builtins__": vars(builtins).copy()}
+ self.tool_names: set[str] = set()
+ self.output_fields: list[dict[str, Any]] | None = None
+
+ def call_tool(self, name: str, *args: Any, **kwargs: Any) -> Any:
+ return self.protocol.call_tool(name, args, kwargs)
+
+ def configure(self, tool_names: list[str], output_fields: list[dict[str, Any]] | None) -> None:
+ if any(
+ not name.isidentifier() or keyword.iskeyword(name) or name in {"SUBMIT", "__builtins__"}
+ for name in tool_names
+ ):
+ raise ValueError("tool names must be non-reserved Python identifiers")
+ if output_fields is not None:
+ names = [field.get("name") for field in output_fields]
+ if any(not isinstance(name, str) or not name.isidentifier() or keyword.iskeyword(name) for name in names):
+ raise ValueError("output field names must be Python identifiers")
+ if len(names) != len(set(names)):
+ raise ValueError("output field names must be unique")
+ for name in self.tool_names:
+ self.namespace.pop(name, None)
+ self.tool_names = set(tool_names)
+ self.output_fields = output_fields
+ for name in tool_names:
+ self.namespace[name] = lambda *args, __name=name, **kwargs: self.call_tool(__name, *args, **kwargs)
+ self.namespace["__builtins__"] = vars(builtins).copy()
+ self.namespace["SUBMIT"] = self.submit
+
+ def submit(self, *args: Any, **kwargs: Any) -> None:
+ if self.output_fields is None:
+ if len(args) != 1 or kwargs:
+ raise TypeError("SUBMIT requires one output value")
+ raise Submission({"output": args[0]})
+ names = [field["name"] for field in self.output_fields]
+ if args and kwargs:
+ raise TypeError("SUBMIT accepts positional or keyword values, not both")
+ values = dict(zip(names, args, strict=False)) if args else dict(kwargs)
+ if set(values) != set(names) or len(args) > len(names):
+ raise TypeError("SUBMIT fields do not match the configured output fields")
+ raise Submission(values)
+
+ def execute(self, request: dict[str, Any]) -> dict[str, Any]:
+ self.configure(request["tools"], request.get("output_fields"))
+ variables = request["variables"]
+ if {"SUBMIT", "__builtins__", *self.tool_names} & variables.keys():
+ raise ValueError("variables cannot replace interpreter-owned globals")
+ code = request["code"]
+ self.namespace.update(variables)
+ outcome: dict[str, Any]
+ with CapturedOutput() as captured:
+ try:
+ tree = ast.parse(code, mode="exec")
+ last = tree.body.pop() if tree.body and isinstance(tree.body[-1], ast.Expr) else None
+ exec(compile(tree, "<interpreter>", "exec"), self.namespace)
+ value = (
+ eval(compile(ast.Expression(last.value), "<interpreter>", "eval"), self.namespace) if last else None
+ )
+ outcome = {"type": "result", "value": jsonable(value)}
+ except Submission as submission:
+ outcome = {"type": "final", "value": jsonable(submission.value)}
+ except SyntaxError as exc:
+ outcome = {"type": "syntax", "error": str(exc)}
+ except BaseException as exc:
+ outcome = {"type": "execution_error", "error": describe(exc)}
+ outcome["stdout"] = captured.value
+ leaked_threads = [thread.name for thread in threading.enumerate() if thread not in self.worker_threads]
+ if leaked_threads:
+ return {
+ "type": "terminal_error",
+ "error": f"executed code left background threads running: {leaked_threads!r}",
+ }
+ return outcome
+
+
+def jsonable(value: Any) -> Any:
+ try:
+ json.dumps(value, allow_nan=False)
+ return value
+ except (TypeError, ValueError):
+ return repr(value)
+
+
+def describe(exc: BaseException) -> str:
+ return f"{type(exc).__name__}: {exc}"
+
+
+def send(message: dict[str, Any]) -> None:
+ with _send_lock:
+ print(json.dumps(message, separators=(",", ":"), allow_nan=False), file=_protocol_output, flush=True)
+
+
+def receive() -> dict[str, Any]:
+ line = sys.__stdin__.readline()
+ if not line:
+ raise EOFError
+ message = json.loads(line)
+ if not isinstance(message, dict) or not isinstance(message.get("type"), str):
+ raise ValueError("protocol messages must be tagged objects")
+ return message
+
+
+def main() -> None:
+ protocol = Protocol()
+ session = Session(protocol)
+ send({"type": "ready"})
+ try:
+ while True:
+ request = protocol.commands.get()
+ if request is None:
+ return
+ if request.get("type") == "shutdown":
+ return
+ if request.get("type") == "protocol_error":
+ raise ValueError(request.get("error"))
+ if request.get("type") != "execute":
+ raise ValueError("unknown request")
+ response = session.execute(request)
+ send(response)
+ if response.get("type") == "terminal_error":
+ return
+ except EOFError:
+ return
+ except BaseException as exc:
+ send({"type": "terminal_error", "error": describe(exc)})
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tests/primitives/test_local_interpreter.py b/tests/primitives/test_local_interpreter.py
new file mode 100644
index 000000000..fd87b7084
--- /dev/null
+++ b/tests/primitives/test_local_interpreter.py
@@ -0,0 +1,350 @@
+import contextvars
+import os
+import tempfile
+import threading
+import time
+from pathlib import Path
+
+import pytest
+
+import dspy
+from dspy.primitives.code_interpreter import CodeExecutionError, CodeInterpreterError, FinalOutput
+from dspy.utils.callback import BaseCallback
+from dspy.utils.dummies import DummyLM
+
+
+def test_persistent_worker_is_separate_and_captures_stdout(capsys):
+ with dspy.LocalInterpreter() as interpreter:
+ assert interpreter.execute("import os\nos.getpid()") != os.getpid()
+ interpreter.execute("remembered = 41")
+ assert interpreter.execute("remembered + 1") == 42
+ assert interpreter.execute("print('guest only')") == "guest only"
+ assert capsys.readouterr().out == ""
+
+
+def test_os_level_output_does_not_corrupt_protocol():
+ with dspy.LocalInterpreter() as interpreter:
+ output = interpreter.execute(
+ "import os, subprocess, sys\n"
+ "os.write(1, b'guest fd stdout\\n')\n"
+ "os.write(2, b'guest fd stderr\\n')\n"
+ "completed = subprocess.run([sys.executable, '-c', \"import os; os.write(1, b'child stdout\\\\n')\"])"
+ )
+ assert output == "guest fd stdout\nguest fd stderr\nchild stdout"
+ assert interpreter.execute("6 * 7") == 42
+
+
+def test_python_stdout_reassignment_does_not_break_later_capture():
+ with dspy.LocalInterpreter() as interpreter:
+ assert interpreter.execute("import io, sys\nsys.stdout = io.StringIO()\nprint('hidden')") is None
+ assert interpreter.execute("print('captured')") == "captured"
+
+
+def test_background_thread_is_terminal_before_it_can_cross_executions():
+ calls = []
+ interpreter = dspy.LocalInterpreter(tools={"record": lambda *, value: calls.append(value)})
+
+ with pytest.raises(CodeInterpreterError, match="left background threads running"):
+ interpreter.execute(
+ "import threading\n"
+ "def late():\n"
+ " threading.Event().wait(60)\n"
+ " record(value='late')\n"
+ "threading.Thread(target=late, daemon=True).start()"
+ )
+
+ assert calls == []
+ with pytest.raises(CodeInterpreterError, match="shut down"):
+ interpreter.execute("1")
+
+
+def test_shutdown_terminates_descendant_processes():
+ fd, path_string = tempfile.mkstemp(prefix="local-interpreter-child-", suffix=".txt")
+ os.close(fd)
+ path = Path(path_string)
+ path.unlink()
+ interpreter = dspy.LocalInterpreter()
+
+ try:
+ assert interpreter.execute(
+ "import subprocess, sys\n"
+ f"path = {str(path)!r}\n"
+ "child = subprocess.Popen([sys.executable, '-c', "
+ "f\"import time; from pathlib import Path; time.sleep(0.4); Path({path!r}).write_text('survived')\"])\n"
+ "'child started'"
+ ) == "child started"
+ interpreter.shutdown()
+ time.sleep(0.8)
+ assert not path.exists()
+ finally:
+ interpreter.shutdown()
+ path.unlink(missing_ok=True)
+
+
+def test_tools_and_typed_submit():
+ fields = [{"name": "answer", "type": "str"}, {"name": "score", "type": "int"}]
+ with dspy.LocalInterpreter(
+ tools={"add": lambda *, left, right: left + right}, output_fields=fields
+ ) as interpreter:
+ assert interpreter.execute("add(left=19, right=23)") == 42
+ result = interpreter.execute("SUBMIT(answer='yes', score=42)")
+ assert isinstance(result, FinalOutput)
+ assert result.output == {"answer": "yes", "score": 42}
+
+
[email protected]("name", ["not-valid", "class", "SUBMIT", "__builtins__"])
+def test_invalid_tool_names_fail_without_starting_or_consuming_session(name):
+ with pytest.raises(ValueError, match="invalid names"):
+ dspy.LocalInterpreter(tools={name: lambda: 1})
+
+ interpreter = dspy.LocalInterpreter()
+ interpreter.tools[name] = lambda: 1
+ with pytest.raises(ValueError, match="invalid names"):
+ interpreter.execute("1")
+ assert interpreter._process is None
+
+ interpreter.tools = {}
+ assert interpreter.execute("6 * 7") == 42
+ interpreter.shutdown()
+
+
+ "output_fields",
+ [
+ [{"name": "class"}],
+ [{"name": "not-valid"}],
+ [{"name": "answer"}, {"name": "answer"}],
+ [{"type": "str"}],
+ [{"name": "answer", "default": object()}],
+ ],
+)
+def test_invalid_output_fields_fail_without_consuming_session(output_fields):
+ with pytest.raises(ValueError):
+ dspy.LocalInterpreter(output_fields=output_fields)
+
+ interpreter = dspy.LocalInterpreter()
+ assert interpreter.execute("remembered = 41") is None
+ process = interpreter._process
+ interpreter.output_fields = output_fields
+ with pytest.raises(ValueError):
+ interpreter.execute("1")
+ assert interpreter._process is process
+
+ interpreter.output_fields = [{"name": "answer"}]
+ assert interpreter.execute("remembered + 1") == 42
+ assert interpreter.execute("SUBMIT(answer=42)").output == {"answer": 42}
+ interpreter.shutdown()
+
+
+def test_reserved_variable_names_fail_without_starting_or_consuming_session():
+ interpreter = dspy.LocalInterpreter(tools={"add": lambda left, right: left + right})
+ for variables in ({"class": 1}, {"SUBMIT": "shadowed"}, {"__builtins__": {}}, {"add": "shadowed"}):
+ with pytest.raises(CodeInterpreterError, match="invalid names"):
+ interpreter.execute("1", variables=variables)
+ assert interpreter._process is None
+
+ assert interpreter.execute("len([1])") == 1
+ assert interpreter.execute("add(19, 23)") == 42
+ assert interpreter.execute("SUBMIT(42)").output == {"output": 42}
+ interpreter.shutdown()
+
+
+def test_guest_cannot_corrupt_interpreter_owned_globals_across_executions():
+ with dspy.LocalInterpreter(tools={"add": lambda left, right: left + right}) as interpreter:
+ interpreter.execute("__builtins__['len'] = None\nSUBMIT = None\nadd = None")
+ assert interpreter.execute("len([1])") == 1
+ assert interpreter.execute("add(19, 23)") == 42
+ assert interpreter.execute("SUBMIT(42)").output == {"output": 42}
+ with pytest.raises(CodeExecutionError, match="_builtins"):
+ interpreter.execute("SUBMIT.__self__._builtins['len'] = None")
+ assert interpreter.execute("len([1])") == 1
+
+
+def test_errors_are_recoverable_but_timeout_is_terminal():
+ interpreter = dspy.LocalInterpreter(execution_timeout=0.1)
+ with pytest.raises(SyntaxError):
+ interpreter.execute("if")
+ with pytest.raises(CodeExecutionError, match="ZeroDivisionError"):
+ interpreter.execute("1 / 0")
+ assert interpreter.execute("6 * 7") == 42
+ with pytest.raises(CodeInterpreterError, match="exceeded execution timeout"):
+ interpreter.execute("import time\ntime.sleep(10)")
+ with pytest.raises(CodeInterpreterError, match="shut down"):
+ interpreter.execute("1")
+ interpreter.shutdown()
+
+
+def test_async_host_tool():
+ async def add(*, left, right):
+ return left + right
+
+ with dspy.LocalInterpreter(tools={"add": add}) as interpreter:
+ assert interpreter.execute("add(left=19, right=23)") == 42
+
+
+def test_timed_async_host_tool_preserves_context():
+ marker = contextvars.ContextVar("marker", default="missing")
+ marker.set("visible")
+
+ async def read_marker():
+ return marker.get()
+
+ with dspy.LocalInterpreter(tools={"read_marker": read_marker}, execution_timeout=1) as interpreter:
+ assert interpreter.execute("read_marker()") == "visible"
+
+
+def test_host_tool_within_execution_timeout():
+ def slow():
+ time.sleep(0.08)
+ return 42
+
+ with dspy.LocalInterpreter(tools={"slow": slow}, execution_timeout=1) as interpreter:
+ assert interpreter.execute("slow()") == 42
+
+
+def test_parallel_guest_code_invokes_host_tools_concurrently_and_routes_responses():
+ rendezvous = threading.Barrier(2)
+
+ def echo(value):
+ rendezvous.wait(timeout=1)
+ if value == "slow":
+ time.sleep(0.05)
+ return value
+
+ code = """
+from concurrent.futures import ThreadPoolExecutor
+with ThreadPoolExecutor(max_workers=2) as pool:
+ futures = [pool.submit(echo, value) for value in ["slow", "fast"]]
+[future.result() for future in futures]
+"""
+ with dspy.LocalInterpreter(tools={"echo": echo}, execution_timeout=2) as interpreter:
+ assert interpreter.execute(code) == ["slow", "fast"]
+
+
+def test_execution_timeout_does_not_wait_for_blocked_host_tool():
+ tool_started = threading.Event()
+
+ def blocked():
+ tool_started.set()
+ time.sleep(1)
+ return 42
+
+ interpreter = dspy.LocalInterpreter(tools={"blocked": blocked}, execution_timeout=0.1)
+ started = time.monotonic()
+ with pytest.raises(CodeInterpreterError, match="exceeded execution timeout"):
+ interpreter.execute("blocked()")
+ elapsed = time.monotonic() - started
+
+ assert tool_started.is_set()
+ assert elapsed < 0.5
+ with pytest.raises(CodeInterpreterError, match="shut down"):
+ interpreter.execute("1")
+
+
+def test_interpreter_callbacks():
+ events = []
+
+ class RecordingCallback(BaseCallback):
+ def on_interpreter_execute_start(self, call_id, instance, inputs):
+ events.append(("execute_start", inputs["code"]))
+
+ def on_interpreter_execute_end(self, call_id, outputs, exception=None):
+ events.append(("execute_end", outputs))
+
+ def on_interpreter_tool_call_start(self, call_id, instance, inputs):
+ events.append(("tool_start", inputs["tool_name"]))
+
+ def on_interpreter_tool_call_end(self, call_id, outputs, exception=None):
+ events.append(("tool_end", outputs))
+
+ with dspy.LocalInterpreter(tools={"answer": lambda: 42}, callbacks=[RecordingCallback()]) as interpreter:
+ assert interpreter.execute("answer()") == 42
+
+ assert ("execute_start", "answer()") in events
+ assert ("tool_start", "answer") in events
+ assert ("tool_end", 42) in events
+ assert ("execute_end", 42) in events
+
+
+def test_rlm_uses_local_interpreter():
+ calls = []
+
+ def add(*, left: int, right: int) -> int:
+ calls.append((left, right))
+ return left + right
+
+ class Actions(dspy.Predict):
+ def __init__(self, signature):
+ super().__init__(signature)
+ self.actions = iter(
+ [
+ dspy.Prediction(reasoning="calculate", code="total = add(left=19, right=23)"),
+ dspy.Prediction(reasoning="submit", code="SUBMIT(answer=str(total))"),
+ ]
+ )
+
+ def forward(self, **kwargs):
+ return next(self.actions)
+
+ rlm = dspy.RLM(
+ "question: str -> answer: str",
+ max_iters=2,
+ tools=[add],
+ interpreter_factory=dspy.LocalInterpreter,
+ )
+ rlm.generate_action = Actions(rlm.generate_action.signature)
+
+ assert rlm(question="What is 19 + 23?").answer == "42"
+ assert calls == [(19, 23)]
+ assert dspy.LocalInterpreter.execution_instructions in rlm.generate_action.signature.instructions
+
+
+async def test_async_rlm_uses_local_interpreter_with_async_tool():
+ async def add(*, left: int, right: int) -> int:
+ return left + right
+
+ class Actions(dspy.Predict):
+ def __init__(self, signature):
+ super().__init__(signature)
+ self.actions = iter(
+ [
+ dspy.Prediction(reasoning="calculate", code="total = add(left=19, right=23)"),
+ dspy.Prediction(reasoning="submit", code="SUBMIT(answer=str(total))"),
+ ]
+ )
+
+ async def aforward(self, **kwargs):
+ return next(self.actions)
+
+ rlm = dspy.RLM(
+ "question: str -> answer: str",
+ max_iters=2,
+ tools=[add],
+ interpreter_factory=dspy.LocalInterpreter,
+ )
+ rlm.generate_action = Actions(rlm.generate_action.signature)
+
+ assert (await rlm.acall(question="What is 19 + 23?")).answer == "42"
+
+
+def test_flex_uses_local_interpreter():
+ class Signature(dspy.Signature):
+ value: int = dspy.InputField()
+ result: int = dspy.OutputField()
+
+ module_src = """
+class AddModule(dspy.Module):
+ def __init__(self):
+ super().__init__()
+ self.solve = dspy.Predict("value: int -> result: int")
+
+ def forward(self, **inputs):
+ predicted = self.solve(value=inputs["value"])
+ return dspy.Prediction(result=predicted.result)
+""".strip()
+
+ program = dspy.Flex(Signature, interpreter_factory=dspy.LocalInterpreter)
+ program._bind_code(module_src)
+ with dspy.context(lm=DummyLM([{"result": 42}])):
+ assert program(value=20).result == 42