What is DSPy?
DSPy is an open-source framework from Stanford University that fundamentally changes how we build LLM applications. Instead of manually crafting prompts, you declare what you want and DSPy automatically optimizes the prompts and fine-tunes models to achieve it. Think of it as a compiler for LLM programs.
"DSPy is a framework for algorithmically optimizing LM prompts and weights, especially when LMs are used one or more times within a pipeline."
Declarative
Define what, not how
Auto-Optimize
Compiles prompts
Modular
Composable modules
Research-Backed
From Stanford NLP
The DSPy Philosophy
Traditional Approach
- • Manually write prompts
- • Trial and error optimization
- • Prompts break with model updates
- • Hard to debug and maintain
- • "Prompt engineering is an art"
DSPy Approach
- • Declare signatures (input → output)
- • Optimizer finds best prompt
- • Recompile for new models
- • Structured, testable code
- • "Programming, not prompting"
Core Concepts
Signatures
Declarative specifications of what a module should do. They define inputs and outputs without specifying how. Like function type hints for LLMs.
# Simple signature syntax: "input -> output"
"question -> answer"
# With multiple fields
"context, question -> reasoning, answer"
# With descriptions (class-based)
class QA(dspy.Signature):
"""Answer questions based on context."""
context = dspy.InputField(desc="The context")
question = dspy.InputField()
answer = dspy.OutputField(desc="Brief answer")Modules
Building blocks that implement prompting techniques (CoT, ReAct, RAG). Like PyTorch modules but for LLM pipelines. Compose them to build complex applications.
Optimizers (Teleprompters)
Algorithms that compile your program by finding optimal prompts, few-shot examples, or fine-tuning weights. Given data and a metric, they optimize automatically.
Quick Start: Simple QA
A minimal example showing DSPy's declarative approach:
# Install DSPy
# pip install dspy
import dspy
# Configure the language model
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)
# Define what we want (signature)
qa = dspy.Predict("question -> answer")
# Use it
result = qa(question="What is the capital of France?")
print(result.answer)Expected Output
"Paris"
Example: Chain of Thought
Use ChainOfThought for
step-by-step reasoning:
import dspy
# Configure LM
lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)
# Define a signature with reasoning
class MathProblem(dspy.Signature):
"""Solve math word problems step by step."""
problem = dspy.InputField(desc="A math word problem")
reasoning = dspy.OutputField(desc="Step-by-step solution")
answer = dspy.OutputField(desc="Final numeric answer")
# Use Chain of Thought module
solver = dspy.ChainOfThought(MathProblem)
# Solve a problem
result = solver(problem="""If a train travels at 60 mph for 2.5 hours,
then slows down to 40 mph for another 1.5 hours,
how far did it travel in total?""")
print("Reasoning:", result.reasoning)
print("Answer:", result.answer)Expected Output
Reasoning: "First, calculate distance at 60 mph for 2.5 hours: 60 × 2.5 = 150 miles. Then, calculate distance at 40 mph for 1.5 hours: 40 × 1.5 = 60 miles. Total: 150 + 60 = 210 miles." Answer: "210 miles"
Example: Auto-Optimization
The real power of DSPy—automatically optimize prompts with data:
import dspy
from dspy.teleprompt import BootstrapFewShot
# Configure
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)
# Define your module
class SentimentClassifier(dspy.Module):
def __init__(self):
self.classify = dspy.ChainOfThought("text -> sentiment")
def forward(self, text):
return self.classify(text=text)
# Training data (can be just 5-10 examples!)
trainset = [
dspy.Example(text="This movie was amazing!", sentiment="positive"),
dspy.Example(text="Terrible service, never again.", sentiment="negative"),
dspy.Example(text="It was okay, nothing special.", sentiment="neutral"),
# ... more examples
]
# Define a metric
def accuracy(example, prediction, trace=None):
return example.sentiment.lower() == prediction.sentiment.lower()
# Compile / Optimize
optimizer = BootstrapFewShot(metric=accuracy)
optimized_classifier = optimizer.compile(
SentimentClassifier(),
trainset=trainset
)
# Now use the optimized version!
result = optimized_classifier(text="Best purchase ever!")
print(result.sentiment) # "positive"What happened? The optimizer analyzed your examples, generated optimal few-shot demonstrations, and created an enhanced prompt—all automatically!
Example: RAG Pipeline
Build a complete RAG pipeline with DSPy:
import dspy
from dspy.retrieve.chromadb_rm import ChromadbRM
# Configure LM and retriever
lm = dspy.LM("openai/gpt-4o-mini")
retriever = ChromadbRM(
collection_name="docs",
persist_directory="./chroma_db",
k=3 # top-k docs
)
dspy.configure(lm=lm, rm=retriever)
# Define RAG module
class RAG(dspy.Module):
def __init__(self):
self.retrieve = dspy.Retrieve(k=3)
self.generate = dspy.ChainOfThought("context, question -> answer")
def forward(self, question):
# Retrieve relevant docs
docs = self.retrieve(question).passages
context = "\n".join(docs)
# Generate answer with context
return self.generate(context=context, question=question)
# Use it
rag = RAG()
answer = rag(question="What is DSPy?")
print(answer.answer)Built-in Modules
dspy.Predict
Basic prediction module. Takes a signature and returns structured output.
dspy.ChainOfThought
Adds step-by-step reasoning before the final answer.
dspy.ReAct
Reason-and-Act pattern for tool-using agents with interleaved reasoning.
dspy.Retrieve
Retrieval module for RAG. Works with various vector stores.
dspy.ProgramOfThought
Generates code to solve problems, then executes it.
dspy.Assert / Suggest
Add constraints and guardrails to module outputs.
DSPy vs Other Frameworks
| Aspect | DSPy | LangChain | LlamaIndex |
|---|---|---|---|
| Philosophy | Program, don't prompt | Chain components | Index and query data |
| Prompt Handling | Auto-optimized | Manual templates | Manual templates |
| Optimization | Built-in compilers | External tools | External tools |
| Learning Curve | Higher (new paradigm) | Lower | Lower |
| Best For | Research, optimization | Rapid prototyping | RAG applications |
When to Use DSPy
Great For
- • Research and experimentation
- • Complex multi-step pipelines
- • When you have evaluation data
- • Reproducible, testable LLM code
- • Portability across models
Less Ideal For
- • Quick prototypes / POCs
- • Simple, one-off prompts
- • When you don't have eval data
- • Teams new to LLMs