DSPy: The framework for programming—not prompting—language models
Python
Run the `typos` tool against the codebase (#8560)
Kurt McKeef132b376aa1b
diff --git a/docs/docs/api/optimizers/MIPROv2.md b/docs/docs/api/optimizers/MIPROv2.mdindex c8669e9ed..3eedf8e5e 100644--- a/docs/docs/api/optimizers/MIPROv2.md+++ b/docs/docs/api/optimizers/MIPROv2.md@@ -64,6 +64,6 @@ These steps are broken down in more detail below:2) **Propose Instruction Candidates**. The instruction proposer includes (1) a generated summary of properties of the training dataset, (2) a generated summary of your LM program's code and the specific predictor that an instruction is being generated for, (3) the previously bootstrapped few-shot examples to show reference inputs / outputs for a given predictor and (4) a randomly sampled tip for generation (i.e. "be creative", "be concise", etc.) to help explore the feature space of potential instructions. This context is provided to a `prompt_model` which writes high quality instruction candidates.-3) **Find an Optimized Combination of Few-Shot Examples & Instructions**. Finally, we use Bayesian Optimization to choose which combinations of instructions and demonstrations work best for each predictor in our program. This works by running a series of `num_trials` trials, where a new set of prompts are evaluated over our validation set at each trial. The new set of prompts are only evaluated on a minibatch of size `minibatch_size` at each trial (when `minibatch`=`True`). The best averaging set of prompts is then evalauted on the full validation set every `minibatch_full_eval_steps`. At the end of the optimization process, the LM program with the set of prompts that performed best on the full validation set is returned.+3) **Find an Optimized Combination of Few-Shot Examples & Instructions**. Finally, we use Bayesian Optimization to choose which combinations of instructions and demonstrations work best for each predictor in our program. This works by running a series of `num_trials` trials, where a new set of prompts are evaluated over our validation set at each trial. The new set of prompts are only evaluated on a minibatch of size `minibatch_size` at each trial (when `minibatch`=`True`). The best averaging set of prompts is then evaluated on the full validation set every `minibatch_full_eval_steps`. At the end of the optimization process, the LM program with the set of prompts that performed best on the full validation set is returned.For those interested in more details, more information on `MIPROv2` along with a study on `MIPROv2` compared with other DSPy optimizers can be found in [this paper](https://arxiv.org/abs/2406.11695).diff --git a/docs/docs/learn/evaluation/overview.md b/docs/docs/learn/evaluation/overview.mdindex 26141880c..1e0e6aaa7 100644--- a/docs/docs/learn/evaluation/overview.md+++ b/docs/docs/learn/evaluation/overview.md@@ -4,7 +4,7 @@ sidebar_position: 1# Evaluation in DSPy-Once you have an initial system, it's time to **collect an initial development set** so you can refine it more systematically. Even 20 input examples of your task can be useful, though 200 goes a long way. Depending on your _metric_, you either just need inputs and no labels at all, or you need inputs and the _final_ outputs of your system. (You almost never need labels for the intermediate steps in your program in DSPy.) You can probably find datasets that are adjacent to your task on, say, HuggingFace datasets or in a naturally occuring source like StackExchange. If there's data whose licenses are permissive enough, we suggest you use them. Otherwise, you can label a few examples by hand or start deploying a demo of your system and collect initial data that way.+Once you have an initial system, it's time to **collect an initial development set** so you can refine it more systematically. Even 20 input examples of your task can be useful, though 200 goes a long way. Depending on your _metric_, you either just need inputs and no labels at all, or you need inputs and the _final_ outputs of your system. (You almost never need labels for the intermediate steps in your program in DSPy.) You can probably find datasets that are adjacent to your task on, say, HuggingFace datasets or in a naturally occurring source like StackExchange. If there's data whose licenses are permissive enough, we suggest you use them. Otherwise, you can label a few examples by hand or start deploying a demo of your system and collect initial data that way.Next, you should **define your DSPy metric**. What makes outputs from your system good or bad? Invest in defining metrics and improving them incrementally over time; it's hard to consistently improve what you aren't able to define. A metric is a function that takes examples from your data and takes the output of your system, and returns a score. For simple tasks, this could be just "accuracy", e.g. for simple classification or short-form QA tasks. For most applications, your system will produce long-form outputs, so your metric will be a smaller DSPy program that checks multiple properties of the output. Getting this right on the first try is unlikely: start with something simple and iterate.diff --git a/docs/docs/tutorials/custom_module/index.ipynb b/docs/docs/tutorials/custom_module/index.ipynbindex 6fd08fdb4..80ad95900 100644--- a/docs/docs/tutorials/custom_module/index.ipynb+++ b/docs/docs/tutorials/custom_module/index.ipynb@@ -70,7 +70,7 @@"id": "DziTWwT8_TrY"},"source": [- "Let's illustrate this with a practical code example. We will build a simple Retrieval-Augmented Generation (RAG) application with mulitple stages:\n",+ "Let's illustrate this with a practical code example. We will build a simple Retrieval-Augmented Generation (RAG) application with multiple stages:\n","\n","1. **Query Generation:** Generate a suitable query based on the user's question to retrieve relevant context.\n","2. **Context Retrieval:** Fetch context using the generated query.\n",diff --git a/docs/docs/tutorials/customer_service_agent/index.ipynb b/docs/docs/tutorials/customer_service_agent/index.ipynbindex bc5ad9068..1e142f079 100644--- a/docs/docs/tutorials/customer_service_agent/index.ipynb+++ b/docs/docs/tutorials/customer_service_agent/index.ipynb@@ -115,7 +115,7 @@"source": ["### Create Dummy Data\n","\n",- "Let's also create some dummy data so that the airline agent can do the work. We need to create a few flights and a few users, and initilize empty dictionaries for the itineraries and custom support tickets."+ "Let's also create some dummy data so that the airline agent can do the work. We need to create a few flights and a few users, and initialize empty dictionaries for the itineraries and custom support tickets."]},{@@ -295,11 +295,11 @@"source": ["import dspy\n","\n",- "class DSPyAirlineCustomerSerice(dspy.Signature):\n",+ "class DSPyAirlineCustomerService(dspy.Signature):\n"," \"\"\"You are an airline customer service agent that helps user book and manage flights.\n","\n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\"\"\"\n",+ " fulfill users' request.\"\"\"\n","\n"," user_request: str = dspy.InputField()\n"," process_result: str = dspy.OutputField(\n",@@ -319,7 +319,7 @@"outputs": [],"source": ["agent = dspy.ReAct(\n",- " DSPyAirlineCustomerSerice,\n",+ " DSPyAirlineCustomerService,\n"," tools = [\n"," fetch_flight_info,\n"," fetch_itinerary,\n",@@ -497,7 +497,7 @@" You are an airline customer service agent that helps user book and manage flights. \n"," \n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\n",+ " fulfill users' request.\n"," \n"," You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n"," Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",@@ -579,7 +579,7 @@" You are an airline customer service agent that helps user book and manage flights. \n"," \n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\n",+ " fulfill users' request.\n"," \n"," You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n"," Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",@@ -672,7 +672,7 @@" You are an airline customer service agent that helps user book and manage flights. \n"," \n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\n",+ " fulfill users' request.\n"," \n"," You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n"," Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",@@ -777,7 +777,7 @@" You are an airline customer service agent that helps user book and manage flights. \n"," \n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\n",+ " fulfill users' request.\n"," \n"," You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n"," Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",@@ -894,7 +894,7 @@" You are an airline customer service agent that helps user book and manage flights. \n"," \n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\n",+ " fulfill users' request.\n"," \n"," You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n"," Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",@@ -1019,7 +1019,7 @@" You are an airline customer service agent that helps user book and manage flights. \n"," \n"," You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",- " fullfil users' request.\n",+ " fulfill users' request.\n","\n","\n","\u001b[31mUser message:\u001b[0m\n",diff --git a/docs/docs/tutorials/games/index.ipynb b/docs/docs/tutorials/games/index.ipynbindex 4046a8abf..354779924 100644--- a/docs/docs/tutorials/games/index.ipynb+++ b/docs/docs/tutorials/games/index.ipynb@@ -376,7 +376,7 @@"data": {"text/plain": ["Prediction(\n",- " trajecotry=['> go to countertop 1', 'You arrive at countertop 1. On the countertop 1, you see a candle 1, a soapbar 1, a soapbottle 2, a soapbottle 1, and a spraybottle 1.', '> take soapbar 1 from countertop 1', 'You pick up the soapbar 1 from the countertop 1.', '> go to garbagecan 1', 'You arrive at garbagecan 1. On the garbagecan 1, you see nothing.', '> move soapbar 1 to garbagecan 1', 'You move the soapbar 1 to the garbagecan 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> take soapbar 1 from garbagecan 1', 'You pick up the soapbar 1 from the garbagecan 1.', '> move soapbar 1 to garbagecan 1', 'You move the soapbar 1 to the garbagecan 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.'],\n",+ " trajectory=['> go to countertop 1', 'You arrive at countertop 1. On the countertop 1, you see a candle 1, a soapbar 1, a soapbottle 2, a soapbottle 1, and a spraybottle 1.', '> take soapbar 1 from countertop 1', 'You pick up the soapbar 1 from the countertop 1.', '> go to garbagecan 1', 'You arrive at garbagecan 1. On the garbagecan 1, you see nothing.', '> move soapbar 1 to garbagecan 1', 'You move the soapbar 1 to the garbagecan 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> take soapbar 1 from garbagecan 1', 'You pick up the soapbar 1 from the garbagecan 1.', '> move soapbar 1 to garbagecan 1', 'You move the soapbar 1 to the garbagecan 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.', '> examine garbagecan 1', 'On the garbagecan 1, you see a soapbar 1.', '> look', 'You are facing the garbagecan 1. Next to it, you see nothing.'],\n"," success=0\n",")"]@@ -724,7 +724,7 @@"data": {"text/plain": ["Prediction(\n",- " trajecotry=['> go to cabinet 1', 'You arrive at cabinet 1. On the cabinet 1, you see nothing.', '> go to dresser 1', 'You arrive at dresser 1. On the dresser 1, you see a book 1, a newspaper 1, a remotecontrol 1, a statue 3, and a television 1.', '> look', 'You are facing the dresser 1. Next to it, you see nothing.', '> go to sidetable 1', 'You arrive at sidetable 1. On the sidetable 1, you see a cellphone 1, and a desklamp 1.', '> go to sidetable 2', 'You arrive at sidetable 2. On the sidetable 2, you see a box 2.', '> take box 2 from sidetable 2', 'You pick up the box 2 from the sidetable 2.', '> go to dresser 1', 'You arrive at dresser 1. On the dresser 1, you see a book 1, a newspaper 1, a remotecontrol 1, a statue 3, and a television 1.', '> move box 2 to dresser 1', 'You move the box 2 to the dresser 1.'],\n",+ " trajectory=['> go to cabinet 1', 'You arrive at cabinet 1. On the cabinet 1, you see nothing.', '> go to dresser 1', 'You arrive at dresser 1. On the dresser 1, you see a book 1, a newspaper 1, a remotecontrol 1, a statue 3, and a television 1.', '> look', 'You are facing the dresser 1. Next to it, you see nothing.', '> go to sidetable 1', 'You arrive at sidetable 1. On the sidetable 1, you see a cellphone 1, and a desklamp 1.', '> go to sidetable 2', 'You arrive at sidetable 2. On the sidetable 2, you see a box 2.', '> take box 2 from sidetable 2', 'You pick up the box 2 from the sidetable 2.', '> go to dresser 1', 'You arrive at dresser 1. On the dresser 1, you see a book 1, a newspaper 1, a remotecontrol 1, a statue 3, and a television 1.', '> move box 2 to dresser 1', 'You move the box 2 to the dresser 1.'],\n"," success=1\n",")"]diff --git a/docs/docs/tutorials/observability/index.md b/docs/docs/tutorials/observability/index.mdindex 488bed258..803a6cb60 100644--- a/docs/docs/tutorials/observability/index.md+++ b/docs/docs/tutorials/observability/index.md@@ -147,7 +147,7 @@ Click on the most recent trace to view its detailed breakdown:Here, you can examine the input and output of every step in your workflow. For example, the screenshot above shows the `retrieve` function's input and output. By inspecting the retriever's output, you can see that it returned outdated information, which is not sufficient to determine which team Shohei Ohtani plays for in June 2025. You can also inspect-other steps, e.g, anguage model's input, output, and configuration.+other steps, e.g, language model's input, output, and configuration.To address the issue of outdated information, you can replace the `retrieve` function with a web search tool powered by [Tavily search](https://www.tavily.com/).diff --git a/docs/docs/tutorials/optimizer_tracking/index.md b/docs/docs/tutorials/optimizer_tracking/index.mdindex 137327027..ecffbfdf6 100644--- a/docs/docs/tutorials/optimizer_tracking/index.md+++ b/docs/docs/tutorials/optimizer_tracking/index.md@@ -10,9 +10,9 @@ Through the autologging capability, MLflow tracks the following information:* Other configuration settings* **Program States**- * Initial instuctions and few-shot examples- * Optimized instuctions and few-shot examples- * Intermediate instuctions and few-shot examples during optimization+ * Initial instructions and few-shot examples+ * Optimized instructions and few-shot examples+ * Intermediate instructions and few-shot examples during optimization* **Datasets*** Training data useddiff --git a/docs/docs/tutorials/rag/index.ipynb b/docs/docs/tutorials/rag/index.ipynbindex 0f4a241a1..e97cdf27d 100644--- a/docs/docs/tutorials/rag/index.ipynb+++ b/docs/docs/tutorials/rag/index.ipynb@@ -942,7 +942,7 @@"[2] «HIGHMEM is a range of kernels memory space, but it is NOT memory you access but its a place where you put what you want to access. A typical 32bit Linux virtual memory map is like: 0x00000000-0xbfffffff: user process (3GB) 0xc0000000-0xffffffff: kernel space (1GB) (CPU-specific vector and whatsoever are ignored here). Linux splits the 1GB kernel space into 2 pieces, LOWMEM and HIGHMEM. The split varies from installation to installation. If an installation chooses, say, 512MB-512MB for LOW and HIGH mems, the 512MB LOWMEM (0xc0000000-0xdfffffff) is statically mapped at the kernel boot time; usually the first so many bytes of the physical memory is used for this so that virtual and physical addresses in this range have a constant offset of, say, 0xc0000000. On the other hand, the latter 512MB (HIGHMEM) has no static mapping (although you could leave pages semi-permanently mapped there, but you must do so explicitly in your driver code). Instead, pages are temporarily mapped and unmapped here so that virtual and physical addresses in this range have no consistent mapping. Typical uses of HIGHMEM include single-time data buffers.»\n","[3] «This is relevant to the Linux kernel; Im not sure how any Unix kernel handles this. The High Memory is the segment of memory that user-space programs can address. It cannot touch Low Memory. Low Memory is the segment of memory that the Linux kernel can address directly. If the kernel must access High Memory, it has to map it into its own address space first. There was a patch introduced recently that lets you control where the segment is. The tradeoff is that you can take addressable memory away from user space so that the kernel can have more memory that it does not have to map before using. Additional resources: http://tldp.org/HOWTO/KernelAnalysis-HOWTO-7.html http://linux-mm.org/HighMemory»\n","[4] «The first reference to turn to is Linux Device Drivers (available both online and in book form), particularly chapter 15 which has a section on the topic. In an ideal world, every system component would be able to map all the memory it ever needs to access. And this is the case for processes on Linux and most operating systems: a 32-bit process can only access a little less than 2^32 bytes of virtual memory (in fact about 3GB on a typical Linux 32-bit architecture). It gets difficult for the kernel, which needs to be able to map the full memory of the process whose system call its executing, plus the whole physical memory, plus any other memory-mapped hardware device. So when a 32-bit kernel needs to map more than 4GB of memory, it must be compiled with high memory support. High memory is memory which is not permanently mapped in the kernels address space. (Low memory is the opposite: it is always mapped, so you can access it in the kernel simply by dereferencing a pointer.) When you access high memory from kernel code, you need to call kmap first, to obtain a pointer from a page data structure (struct page). Calling kmap works whether the page is in high or low memory. There is also kmap_atomic which has added constraints but is more efficient on multiprocessor machines because it uses finer-grained locking. The pointer obtained through kmap is a resource: it uses up address space. Once youve finished with it, you must call kunmap (or kunmap_atomic) to free that resource; then the pointer is no longer valid, and the contents of the page cant be accessed until you call kmap again.»\n",- "[5] «/proc/meminfo will tell you how free works, but /proc/kcore can tell you what the kernel uses. From the same page: /proc/kcore This file represents the physical memory of the system and is stored in the ELF core file format. With this pseudo-file, and an unstripped kernel (/usr/src/linux/vmlinux) binary, GDB can be used to examine the current state of any kernel data structures. The total length of the file is the size of physical memory (RAM) plus 4KB. /proc/meminfo This file reports statistics about memory usage on the system. It is used by free(1) to report the amount of free and used memory (both physical and swap) on the system as well as the shared memory and buffers used by the kernel. Each line of the file consists of a parameter name, followed by a colon, the value of the parameter, and an option unit of measurement (e.g., kB). The list below describes the parameter names and the format specifier required to read the field value. Except as noted below, all of the fields have been present since at least Linux 2.6.0. Some fileds are displayed only if the kernel was configured with various options; those dependencies are noted in the list. MemTotal %lu Total usable RAM (i.e., physical RAM minus a few reserved bits and the kernel binary code). MemFree %lu The sum of LowFree+HighFree. Buffers %lu Relatively temporary storage for raw disk blocks that shouldnt get tremendously large (20MB or so). Cached %lu In-memory cache for files read from the disk (the page cache). Doesnt include SwapCached. SwapCached %lu Memory that once was swapped out, is swapped back in but still also is in the swap file. (If memory pressure is high, these pages dont need to be swapped out again because they are already in the swap file. This saves I/O.) Active %lu Memory that has been used more recently and usually not reclaimed unless absolutely necessary. Inactive %lu Memory which has been less recently used. It is more eligible to be reclaimed for other purposes. Active(anon) %lu (since Linux 2.6.28) [To be documented.] Inactive(anon) %lu (since Linux 2.6.28) [To be documented.] Active(file) %lu (since Linux 2.6.28) [To be documented.] Inactive(file) %lu (since Linux 2.6.28) [To be documented.] Unevictable %lu (since Linux 2.6.28) (From Linux 2.6.28 to 2.6.30, CONFIG_UNEVICTABLE_LRU was required.) [To be documented.] Mlocked %lu (since Linux 2.6.28) (From Linux 2.6.28 to 2.6.30, CONFIG_UNEVICTABLE_LRU was required.) [To be documented.] HighTotal %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Total amount of highmem. Highmem is all memory above ~860MB of physical memory. Highmem areas are for use by user-space programs, or for the page cache. The kernel must use tricks to access this memory, making it slower to access than lowmem. HighFree %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Amount of free highmem. LowTotal %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Total amount of lowmem. Lowmem is memory which can be used for everything that highmem can be used for, but it is also available for the kernels use for its own data structures. Among many other things, it is where everything from Slab is allocated. Bad things happen when youre out of lowmem. LowFree %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Amount of free lowmem. MmapCopy %lu (since Linux 2.6.29) (CONFIG_MMU is required.) [To be documented.] SwapTotal %lu Total amount of swap space available. SwapFree %lu Amount of swap space that is currently unused. Dirty %lu Memory which is waiting to get written back to the disk. Writeback %lu Memory which is actively being written back to the disk. AnonPages %lu (since Linux 2.6.18) Non-file backed pages mapped into user-space page tables. Mapped %lu Files which have been mmaped, such as libraries. Shmem %lu (since Linux 2.6.32) [To be documented.] Slab %lu In-kernel data structures cache. SReclaimable %lu (since Linux 2.6.19) Part of Slab, that might be reclaimed, such as caches. SUnreclaim %lu (since Linux 2.6.19) Part of Slab, that cannot be reclaimed on memory pressure. KernelStack %lu (since Linux 2.6.32) Amount of memory allocated to kernel stacks. PageTables %lu (since Linux 2.6.18) Amount of memory dedicated to the lowest level of page tables. Quicklists %lu (since Linux 2.6.27) (CONFIG_QUICKLIST is required.) [To be documented.] NFS_Unstable %lu (since Linux 2.6.18) NFS pages sent to the server, but not yet committed to stable storage. Bounce %lu (since Linux 2.6.18) Memory used for block device bounce buffers. WritebackTmp %lu (since Linux 2.6.26) Memory used by FUSE for temporary writeback buffers. CommitLimit %lu (since Linux 2.6.10) Based on the overcommit ratio (vm.overcommit_ratio), this is the total amount of memory currently available to be allocated on the system. This limit is adhered to only if strict overcommit accounting is enabled (mode 2 in /proc/sys/vm/overcommit_ratio). The CommitLimit is calculated using the following formula: CommitLimit = ([total RAM pages] - [total huge TLB pages]) * overcommit_ratio / 100 + [total swap pages] For example, on a system with 1GB of physical RAM and 7GB of swap with a overcommit_ratio of 30, this formula yields a CommitLimit of 7.3GB. For more details, see the memory overcommit documentation in the kernel source file Documentation/vm/overcommit-accounting. Committed_AS %lu The amount of memory presently allocated on the system. The committed memory is a sum of all of the memory which has been allocated by processes, even if it has not been used by them as of yet. A process which allocates 1GB of memory (using malloc(3) or similar), but touches only 300MB of that memory will show up as using only 300MB of memory even if it has the address space allocated for the entire 1GB. This 1GB is memory which has been committed to by the VM and can be used at any time by the allocating application. With strict overcommit enabled on the system (mode 2 /proc/sys/vm/overcommit_memory), allocations w»\n",+ "[5] «/proc/meminfo will tell you how free works, but /proc/kcore can tell you what the kernel uses. From the same page: /proc/kcore This file represents the physical memory of the system and is stored in the ELF core file format. With this pseudo-file, and an unstripped kernel (/usr/src/linux/vmlinux) binary, GDB can be used to examine the current state of any kernel data structures. The total length of the file is the size of physical memory (RAM) plus 4KB. /proc/meminfo This file reports statistics about memory usage on the system. It is used by free(1) to report the amount of free and used memory (both physical and swap) on the system as well as the shared memory and buffers used by the kernel. Each line of the file consists of a parameter name, followed by a colon, the value of the parameter, and an option unit of measurement (e.g., kB). The list below describes the parameter names and the format specifier required to read the field value. Except as noted below, all of the fields have been present since at least Linux 2.6.0. Some fields are displayed only if the kernel was configured with various options; those dependencies are noted in the list. MemTotal %lu Total usable RAM (i.e., physical RAM minus a few reserved bits and the kernel binary code). MemFree %lu The sum of LowFree+HighFree. Buffers %lu Relatively temporary storage for raw disk blocks that shouldnt get tremendously large (20MB or so). Cached %lu In-memory cache for files read from the disk (the page cache). Doesnt include SwapCached. SwapCached %lu Memory that once was swapped out, is swapped back in but still also is in the swap file. (If memory pressure is high, these pages dont need to be swapped out again because they are already in the swap file. This saves I/O.) Active %lu Memory that has been used more recently and usually not reclaimed unless absolutely necessary. Inactive %lu Memory which has been less recently used. It is more eligible to be reclaimed for other purposes. Active(anon) %lu (since Linux 2.6.28) [To be documented.] Inactive(anon) %lu (since Linux 2.6.28) [To be documented.] Active(file) %lu (since Linux 2.6.28) [To be documented.] Inactive(file) %lu (since Linux 2.6.28) [To be documented.] Unevictable %lu (since Linux 2.6.28) (From Linux 2.6.28 to 2.6.30, CONFIG_UNEVICTABLE_LRU was required.) [To be documented.] Mlocked %lu (since Linux 2.6.28) (From Linux 2.6.28 to 2.6.30, CONFIG_UNEVICTABLE_LRU was required.) [To be documented.] HighTotal %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Total amount of highmem. Highmem is all memory above ~860MB of physical memory. Highmem areas are for use by user-space programs, or for the page cache. The kernel must use tricks to access this memory, making it slower to access than lowmem. HighFree %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Amount of free highmem. LowTotal %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Total amount of lowmem. Lowmem is memory which can be used for everything that highmem can be used for, but it is also available for the kernels use for its own data structures. Among many other things, it is where everything from Slab is allocated. Bad things happen when you're out of lowmem. LowFree %lu (Starting with Linux 2.6.19, CONFIG_HIGHMEM is required.) Amount of free lowmem. MmapCopy %lu (since Linux 2.6.29) (CONFIG_MMU is required.) [To be documented.] SwapTotal %lu Total amount of swap space available. SwapFree %lu Amount of swap space that is currently unused. Dirty %lu Memory which is waiting to get written back to the disk. Writeback %lu Memory which is actively being written back to the disk. AnonPages %lu (since Linux 2.6.18) Non-file backed pages mapped into user-space page tables. Mapped %lu Files which have been mapped, such as libraries. Shmem %lu (since Linux 2.6.32) [To be documented.] Slab %lu In-kernel data structures cache. SReclaimable %lu (since Linux 2.6.19) Part of Slab, that might be reclaimed, such as caches. SUnreclaim %lu (since Linux 2.6.19) Part of Slab, that cannot be reclaimed on memory pressure. KernelStack %lu (since Linux 2.6.32) Amount of memory allocated to kernel stacks. PageTables %lu (since Linux 2.6.18) Amount of memory dedicated to the lowest level of page tables. Quicklists %lu (since Linux 2.6.27) (CONFIG_QUICKLIST is required.) [To be documented.] NFS_Unstable %lu (since Linux 2.6.18) NFS pages sent to the server, but not yet committed to stable storage. Bounce %lu (since Linux 2.6.18) Memory used for block device bounce buffers. WritebackTmp %lu (since Linux 2.6.26) Memory used by FUSE for temporary writeback buffers. CommitLimit %lu (since Linux 2.6.10) Based on the overcommit ratio (vm.overcommit_ratio), this is the total amount of memory currently available to be allocated on the system. This limit is adhered to only if strict overcommit accounting is enabled (mode 2 in /proc/sys/vm/overcommit_ratio). The CommitLimit is calculated using the following formula: CommitLimit = ([total RAM pages] - [total huge TLB pages]) * overcommit_ratio / 100 + [total swap pages] For example, on a system with 1GB of physical RAM and 7GB of swap with a overcommit_ratio of 30, this formula yields a CommitLimit of 7.3GB. For more details, see the memory overcommit documentation in the kernel source file Documentation/vm/overcommit-accounting. Committed_AS %lu The amount of memory presently allocated on the system. The committed memory is a sum of all of the memory which has been allocated by processes, even if it has not been used by them as of yet. A process which allocates 1GB of memory (using malloc(3) or similar), but touches only 300MB of that memory will show up as using only 300MB of memory even if it has the address space allocated for the entire 1GB. This 1GB is memory which has been committed to by the VM and can be used at any time by the allocating application. With strict overcommit enabled on the system (mode 2 /proc/sys/vm/overcommit_memory), allocations w»\n","\n","[[ ## question ## ]]\n","what are high memory and low memory on linux?\n",diff --git a/docs/docs/tutorials/streaming/index.md b/docs/docs/tutorials/streaming/index.mdindex 90e10176b..8782424d4 100644--- a/docs/docs/tutorials/streaming/index.md+++ b/docs/docs/tutorials/streaming/index.md@@ -69,7 +69,7 @@ this example it is the `answer` field. `Prediction` is the program's final outpuimplemented in a sidecar fashion: we enable streaming on the LM so that LM outputs a stream of tokens. We send thesetokens to a side channel, which is being continuously read by the user-defined listeners. Listeners keep interpretingthe stream, and decides if the `signature_field_name` it is listening to has started to appear and has finalized.-Once it decides that the field appears, the listener begins outputing tokens to the async generator users can+Once it decides that the field appears, the listener begins outputting tokens to the async generator users canread. Listeners' internal mechanism changes according to the adapter behind the scene, and because usuallywe cannot decide if a field has finalized until seeing the next field, the listener buffers the output tokensbefore sending to the final generator, which is why you will usually see the last chunk of type `StreamResponse`diff --git a/dspy/adapters/types/base_type.py b/dspy/adapters/types/base_type.pyindex 35334cf76..1a47eb2b4 100644--- a/dspy/adapters/types/base_type.py+++ b/dspy/adapters/types/base_type.py@@ -42,7 +42,7 @@ class Type(pydantic.BaseModel):have arbitrary level of nesting. For example, we detect `Tool` is in `list[dict[str, Tool]]`."""# Direct match. Nested type like `list[dict[str, Event]]` passes `isinstance(annotation, type)` in python 3.10- # while fails in python 3.11. To accomodate users using python 3.10, we need to capture the error and ignore it.+ # while fails in python 3.11. To accommodate users using python 3.10, we need to capture the error and ignore it.try:if isinstance(annotation, type) and issubclass(annotation, cls):return [annotation]diff --git a/dspy/clients/openai.py b/dspy/clients/openai.pyindex c740ed19d..acced217f 100644--- a/dspy/clients/openai.py+++ b/dspy/clients/openai.py@@ -99,7 +99,7 @@ class OpenAIProvider(Provider):if model in _OPENAI_MODELS:return True- # Check if the model is a fine-tuned OpneAI model. Fine-tuned OpenAI+ # Check if the model is a fine-tuned OpenAI model. Fine-tuned OpenAI# models have the prefix "ft:<BASE_MODEL_NAME>:", followed by a string# specifying the fine-tuned model. The following RegEx pattern is used# to match the base model name.diff --git a/dspy/datasets/alfworld/base_config.yml b/dspy/datasets/alfworld/base_config.ymlindex 03c6d146b..d26497016 100644--- a/dspy/datasets/alfworld/base_config.yml+++ b/dspy/datasets/alfworld/base_config.yml@@ -86,7 +86,7 @@ rl:training:max_nb_steps_per_episode: 50 # terminate after this many steps- learn_start_from_this_episode: 0 # delay updates until this epsiode+ learn_start_from_this_episode: 0 # delay updates until this episodetarget_net_update_frequency: 500 # sync target net with online net per this many epochsreplay:diff --git a/dspy/predict/code_act.py b/dspy/predict/code_act.pyindex f880917c4..6d1616e5f 100644--- a/dspy/predict/code_act.py+++ b/dspy/predict/code_act.py@@ -88,7 +88,7 @@ class CodeAct(ReAct, ProgramOfThought):return instructionsdef forward(self, **kwargs):- # Define the tool funcitons in the interpreter+ # Define the tool functions in the interpreterfor tool in self.tools.values():self.interpreter(inspect.getsource(tool.func))diff --git a/dspy/predict/predict.py b/dspy/predict/predict.pyindex a893f0d66..c120a624d 100644--- a/dspy/predict/predict.py+++ b/dspy/predict/predict.py@@ -131,7 +131,7 @@ class Predict(Module, Parameter):and "content" in kwargs["prediction"]):# If the `prediction` is the standard predicted outputs format- # (https://platform.openai.com/docs/guides/predicted-outputs), we remvoe it from input kwargs and add it+ # (https://platform.openai.com/docs/guides/predicted-outputs), we remove it from input kwargs and add it# to the lm kwargs.config["prediction"] = kwargs.pop("prediction")diff --git a/dspy/predict/react.py b/dspy/predict/react.pyindex 42c259c14..5f87879f8 100644--- a/dspy/predict/react.py+++ b/dspy/predict/react.py@@ -210,7 +210,7 @@ One way to fix this is to support `format=fn` in the dspy.InputField() for "trajmeans that care must be taken that the adapter is accessed at `forward` runtime, not signature definition time.Another potential fix is to more natively support a "variadic" input field, where the input is a list of dictionaries,-or a big dictionary, and have each adatper format it accordingly.+or a big dictionary, and have each adapter format it accordingly.Trajectories also affect meta-programming modules that view the trace later. It's inefficient O(n^2) to view thetrace of every module repeating the prefix.diff --git a/dspy/primitives/runner.js b/dspy/primitives/runner.jsindex 4b10bf547..d74653fcf 100644--- a/dspy/primitives/runner.js+++ b/dspy/primitives/runner.js@@ -169,7 +169,7 @@ sys.stderr = old_stderr// Only python exceptions have args.const last_exception_args = pyodide.globals.get("last_exception_args");// Regarding https://pyodide.org/en/stable/usage/type-conversions.html#type-translations-errors,- // we do a addtional `json.dumps` and `JSON.parse` on the values, to avoid the possible memory leak.+ // we do a additional `json.dumps` and `JSON.parse` on the values, to avoid the possible memory leak.errorArgs = JSON.parse(last_exception_args()) || [];}diff --git a/dspy/propose/grounded_proposer.py b/dspy/propose/grounded_proposer.pyindex 880d045d4..13722b4bb 100644--- a/dspy/propose/grounded_proposer.py+++ b/dspy/propose/grounded_proposer.py@@ -347,7 +347,7 @@ class GroundedProposer(Proposer):if self.verbose:print("No demo candidates provided. Running without task demos.")self.use_task_demos = False- # When no demo candidates are provided, defailt to N+ # When no demo candidates are provided, default to Nnum_demos = Nelse:num_demos = max(len(demo_candidates[0]), 1)diff --git a/dspy/propose/utils.py b/dspy/propose/utils.pyindex 454b47fc4..8bd720a23 100644--- a/dspy/propose/utils.py+++ b/dspy/propose/utils.py@@ -7,7 +7,7 @@ import dspytry:from IPython.core.magics.code import extract_symbolsexcept ImportError:- # Won't be able to read code from juptyer notebooks+ # Won't be able to read code from jupyter notebooksextract_symbols = Nonefrom dspy.predict.parameter import Parameterdiff --git a/dspy/streaming/streaming_listener.py b/dspy/streaming/streaming_listener.pyindex 851a79193..59168a7b4 100644--- a/dspy/streaming/streaming_listener.py+++ b/dspy/streaming/streaming_listener.py@@ -104,7 +104,7 @@ class StreamListener:if chunk_message and start_identifier in chunk_message:# If the cache is hit, the chunk_message could be the full response. When it happens we can# directly end the stream listening. In some models like gemini, each stream chunk can be multiple- # tokens, so it's posible that response only has one chunk, we also fall back to this logic.+ # tokens, so it's possible that response only has one chunk, we also fall back to this logic.message_after_start_identifier = chunk_message[chunk_message.find(start_identifier) + len(start_identifier) :]diff --git a/dspy/teleprompt/bettertogether.py b/dspy/teleprompt/bettertogether.pyindex 6c61974df..d1154f9ae 100644--- a/dspy/teleprompt/bettertogether.py+++ b/dspy/teleprompt/bettertogether.py@@ -34,7 +34,7 @@ class BetterTogether(Teleprompter):# TODO: Note that the BetterTogether optimizer is meaningful when# BootstrapFinetune uses a metric to filter the training data before# fine-tuning. However, one can also choose to run this optimizer with- # a BoostrapFinetune without a metric, say, if there aren't labels+ # a BootstrapFinetune without a metric, say, if there aren't labels# available for the training data. Should this be noted somewhere?# TODO: We should re-consider if the metric should be required.self.prompt_optimizer = prompt_optimizer if prompt_optimizer else BootstrapFewShotWithRandomSearch(metric=metric)diff --git a/dspy/teleprompt/mipro_optimizer_v2.py b/dspy/teleprompt/mipro_optimizer_v2.pyindex cf5485460..7dae8bf4f 100644--- a/dspy/teleprompt/mipro_optimizer_v2.py+++ b/dspy/teleprompt/mipro_optimizer_v2.py@@ -135,7 +135,7 @@ class MIPROv2(Teleprompter):# If auto is provided, and either num_candidates or num_trials is not None, raise an errorif self.auto is not None and (self.num_candidates is not None or num_trials is not None):raise ValueError(- "If auto is not None, num_candidates and num_trials cannot be set, since they would be overrided by the auto settings. Please either set auto to None, or do not specify num_candidates and num_trials."+ "If auto is not None, num_candidates and num_trials cannot be set, since they would be overridden by the auto settings. Please either set auto to None, or do not specify num_candidates and num_trials.")# Set random seedsdiff --git a/dspy/teleprompt/utils.py b/dspy/teleprompt/utils.pyindex 4d71259e1..8a5111902 100644--- a/dspy/teleprompt/utils.py+++ b/dspy/teleprompt/utils.py@@ -11,7 +11,7 @@ import numpy as nptry:from IPython.core.magics.code import extract_symbolsexcept ImportError:- # Won't be able to read code from juptyer notebooks+ # Won't be able to read code from jupyter notebooksextract_symbols = Noneimport dspydiff --git a/tests/adapters/test_json_adapter.py b/tests/adapters/test_json_adapter.pyindex 893dcd30d..632dc4d17 100644--- a/tests/adapters/test_json_adapter.py+++ b/tests/adapters/test_json_adapter.py@@ -147,7 +147,7 @@ def test_json_adapter_on_pydantic_model():choices=[Choices(message=Message(- content="{'answer': {'analysis': 'Paris is the captial of France', 'result': 'Paris'}}"+ content="{'answer': {'analysis': 'Paris is the capital of France', 'result': 'Paris'}}"))],@@ -204,7 +204,7 @@ def test_json_adapter_on_pydantic_model():assert expected_input_data in user_message_content# Assert that the adapter output has expected fields and values- assert result.answer.analysis == "Paris is the captial of France"+ assert result.answer.analysis == "Paris is the capital of France"assert result.answer.result == "Paris"@@ -525,7 +525,7 @@ async def test_json_adapter_on_pydantic_model_async():choices=[Choices(message=Message(- content="{'answer': {'analysis': 'Paris is the captial of France', 'result': 'Paris'}}"+ content="{'answer': {'analysis': 'Paris is the capital of France', 'result': 'Paris'}}"))],@@ -584,7 +584,7 @@ async def test_json_adapter_on_pydantic_model_async():assert expected_input_data in user_message_content# Assert that the adapter output has expected fields and values- assert result.answer.analysis == "Paris is the captial of France"+ assert result.answer.analysis == "Paris is the capital of France"assert result.answer.result == "Paris"diff --git a/tests/primitives/test_base_module.py b/tests/primitives/test_base_module.pyindex 716a62687..b78a04d66 100644--- a/tests/primitives/test_base_module.py+++ b/tests/primitives/test_base_module.py@@ -371,7 +371,7 @@ def test_module_history():with patch("litellm.completion") as mock_completion:mock_completion.return_value = ModelResponse(choices=[- Choices(message=Message(content="{'reasoning': 'Paris is the captial of France', 'answer': 'Paris'}"))+ Choices(message=Message(content="{'reasoning': 'Paris is the capital of France', 'answer': 'Paris'}"))],model="openai/gpt-4o-mini",)@@ -390,7 +390,7 @@ def test_module_history():# The same history entity is shared across all the ancestor callers to reduce memory usage.assert id(program.history[0]) == id(program.cot.history[0])- assert program.history[0]["outputs"] == ["{'reasoning': 'Paris is the captial of France', 'answer': 'Paris'}"]+ assert program.history[0]["outputs"] == ["{'reasoning': 'Paris is the capital of France', 'answer': 'Paris'}"]dspy.settings.configure(disable_history=True)@@ -452,7 +452,7 @@ async def test_module_history_async():with patch("litellm.acompletion") as mock_completion:mock_completion.return_value = ModelResponse(choices=[- Choices(message=Message(content="{'reasoning': 'Paris is the captial of France', 'answer': 'Paris'}"))+ Choices(message=Message(content="{'reasoning': 'Paris is the capital of France', 'answer': 'Paris'}"))],model="openai/gpt-4o-mini",)@@ -471,7 +471,7 @@ async def test_module_history_async():# The same history entity is shared across all the ancestor callers to reduce memory usage.assert id(program.history[0]) == id(program.cot.history[0])- assert program.history[0]["outputs"] == ["{'reasoning': 'Paris is the captial of France', 'answer': 'Paris'}"]+ assert program.history[0]["outputs"] == ["{'reasoning': 'Paris is the capital of France', 'answer': 'Paris'}"]with dspy.context(disable_history=True, lm=dspy.LM("openai/gpt-4o-mini", cache=False), adapter=dspy.JSONAdapter()diff --git a/tests/propose/test_grounded_proposer.py b/tests/propose/test_grounded_proposer.pyindex 517f4e174..252afe8ad 100644--- a/tests/propose/test_grounded_proposer.py+++ b/tests/propose/test_grounded_proposer.py@@ -14,7 +14,7 @@ from dspy.utils.dummies import DummyLM],)def test_propose_instructions_for_program(demo_candidates):- # Set large numner here so that lm always returns the same response+ # Set large number here so that lm always returns the same responseprompt_model = DummyLM([{"proposed_instruction": "instruction"}] * 10)program = Predict("question -> answer")trainset = []diff --git a/tests/reliability/README.md b/tests/reliability/README.mdindex faaa810ca..a90265d9b 100644--- a/tests/reliability/README.md+++ b/tests/reliability/README.md@@ -1,6 +1,6 @@# DSPy Reliability Tests-This directory contains reliability tests for DSPy programs. The purpose of these tests is to verify that DSPy programs reliabily produce expected outputs across multiple large language models (LLMs), regardless of model size or capability. These tests are designed to ensure that DSPy programs maintain robustness and accuracy across diverse LLM configurations.+This directory contains reliability tests for DSPy programs. The purpose of these tests is to verify that DSPy programs reliably produce expected outputs across multiple large language models (LLMs), regardless of model size or capability. These tests are designed to ensure that DSPy programs maintain robustness and accuracy across diverse LLM configurations.### Overview