DSPy: The framework for programming—not prompting—language models
Python
Update README.md
Omar Khattab6bfaa2207d8a
diff --git a/README.md b/README.mdindex 0433c1a34..4a1711c83 100644--- a/README.md+++ b/README.md@@ -1,10 +1,12 @@# 🎓𝗗𝗦𝗣: The Demonstrate–Search–Predict Framework-The **DSP** framework provides a programming abstraction for rapidly building reliable AI systems with language models (LMs) and retrieval models (RMs). You write a **DSP program** in a few lines of code, which describes at high level how the problem you'd like to tackle (e.g., answering complex questions about the world) should be _decomposed_ into smaller _transformations_. Your program implements a sophisticated pipeline composed of such transformations (e.g., `generate a search query to find missing information` or `answer this question using the supplied context`), which the LM or RM can handle much more reliably than the overarching problem. Our [research paper](https://arxiv.org/abs/2212.14024) show this can easily lead to up to 120% gains over GPT-3.+The **DSP** framework provides a programming abstraction for _rapidly building sophisticated AI systems_. It's primarily (but not exclusively) designed for tasks that are knowledge intensive (e.g., answering user questions or researching complex topics).-Unlike vanilla in-context learning, you do not hard-code any few-shot prompts in **DSP**. We discourage "prompt engineering" and view it much the same way as hyperparameter tuning in traditional ML: it's a final (and relatively minor) step that's best done _after_ building up an effective architecture and connecting its components together.+You write a **DSP program** in a few lines of code, describing at high level how the task you'd like to tackle should be _decomposed_ into smaller _transformations_. Transformations invoke a language model (LM) and/or a retrieval model (RM) to conduct steps like `generate a search query to find missing information` or `answer this question using the supplied context`. Our [research paper](https://arxiv.org/abs/2212.14024) show that building NLP systems in this way can easily outperform GPT-3.5 by up to 120%.-Instead, **DSP** offers a number of powerful primitives and systematic design patterns for building these architectures — using LMs and search. The **DSP** primitives get the transformations you build working together on your behalf. For instance, **DSP** *annotates* few-shot demonstrations for the LM calls within your arbitrary pipeline automatically, and uses them to improve the quality of your transformations. Once you're happy with things, **DSP** can *compile* your program into a much smaller LM that's much cheaper to deploy.+In **DSP**, developers focus on the _what_ (i.e., the algorithmic design of decomposing the problem) rather than _how_ the transformations are mapped to LM (or RM) calls. In particular, **DSP** discourages "prompt engineering", which we view much the same way as hyperparameter tuning in traditional ML: a final and minor step that's best done _after_ building up an effective architecture.++Instead, **DSP** offers a number of powerful primitives and systematic design patterns for building architectures that orchasterate LMs and RMs. The **DSP** runtime deals with mapping the transformations you build into an effective implementation. For instance, **DSP** *annotates* few-shot demonstrations for the LM calls within your arbitrary pipeline automatically, and uses them to improve the quality of your transformations. Once you're happy with things, **DSP** can *compile* your program into a much cheaper version in which LM calls are transparently replaced with calls to a tiny LM created automatically by the **DSP** runtime.<p align="center">