LLM Evaluation

Testing, Benchmarking & Continuous Quality Assessment for LLM Applications

What is LLM Evaluation?

LLM Evaluation (or "Evals") is the process of systematically testing and measuring the quality, accuracy, safety, and performance of LLM-powered applications. Unlike traditional software testing, evals must account for the non-deterministic nature of LLMs and assess qualities like coherence, helpfulness, and factual accuracy.

"Evals are the key to building reliable AI systems. Without rigorous evaluation, you're essentially shipping untested code. The challenge is that LLM outputs are probabilistic, so your testing strategy must adapt accordingly."

— LLM Engineering Best Practices

Accuracy

Correct answers

Safety

Harmful content

Helpfulness

User satisfaction

Performance

Latency, cost

Types of Evaluation

Ground Truth Evaluation

Compare LLM outputs against known correct answers. Works for factual Q&A, classification, and structured outputs where a "right answer" exists.

Deterministic Automated Needs labeled data

LLM-as-Judge

Use a powerful LLM (like GPT-4) to evaluate outputs from the target model. Great for subjective qualities like helpfulness, coherence, and tone.

Scalable Nuanced Cost Popular

Human Evaluation

Human raters assess outputs for quality, accuracy, and appropriateness. Gold standard for subjective evaluation but expensive and slow.

Gold standard Expensive Slow

Metric-Based Evaluation

Use NLP metrics like BLEU, ROUGE, semantic similarity to compare outputs quantitatively. Best for summarization, translation, and similarity tasks.

Automated Fast Limited scope

Common Evaluation Metrics

Metric	What It Measures	Best For
Exact Match	Output matches expected exactly	Classification, extraction
Contains / Regex	Output contains expected pattern	Keyword presence, format validation
Semantic Similarity	Embedding distance between outputs	Paraphrase detection, meaning
BLEU / ROUGE	N-gram overlap with reference	Translation, summarization
Factual Accuracy	Claims verified against sources	RAG, Q&A systems
Toxicity Score	Harmful content detection	Safety evaluation
LLM Judge Score	GPT-4/Claude rates on criteria	Subjective quality

LLM-as-Judge Implementation

# LLM-as-Judge evaluation
from openai import OpenAI

client = OpenAI()

JUDGE_PROMPT = """You are an expert evaluator. Rate the following response 
on a scale of 1-5 for each criterion.

Question: {question}
Response: {response}

Evaluate on:
1. **Accuracy** (1-5): Is the information factually correct?
2. **Helpfulness** (1-5): Does it address the user's needs?
3. **Clarity** (1-5): Is it well-organized and easy to understand?
4. **Completeness** (1-5): Does it fully answer the question?

Return JSON: {"accuracy": N, "helpfulness": N, "clarity": N, "completeness": N, "reasoning": "..."}
"""

def evaluate_response(question: str, response: str) -> dict:
    """Use GPT-4 to evaluate a response"""
    
    result = client.chat.completions.create(
        model="gpt-4-turbo",
        messages=[{
            "role": "user",
            "content": JUDGE_PROMPT.format(question=question, response=response)
        }],
        response_format={"type": "json_object"}
    )
    
    import json
    scores = json.loads(result.choices[0].message.content)
    scores["average"] = sum([
        scores["accuracy"], scores["helpfulness"], 
        scores["clarity"], scores["completeness"]
    ]) / 4
    
    return scores

Evaluation Pipeline

Test Dataset

Run LLM

Evaluate

Aggregate

Report

Evaluation Frameworks & Tools

LangSmith

LangChain's platform for tracing, evaluating, and monitoring LLM apps with built-in eval tools.

View LangSmith →

OpenAI Evals

OpenAI's open-source framework for evaluating LLMs with extensible eval templates.

View on GitHub →

Braintrust

Enterprise eval platform with logging, scoring, dataset management, and CI/CD integration.

View Braintrust →

RAGAS

Framework specifically for evaluating RAG pipelines with metrics like faithfulness and relevancy.

View RAGAS →

DeepEval

Unit testing framework for LLMs with pytest integration and multiple eval metrics.

View DeepEval →

PromptFoo

CLI tool for testing and comparing prompts with assertions, grading, and CI support.

View PromptFoo →

Best Practices

Do This

Create domain-specific eval datasets
Run evals before every deployment
Combine multiple eval methods
Track metrics over time
Include edge cases and adversarial inputs
Validate LLM-as-Judge with human ratings

Avoid This

Deploying without running evals
Using only exact match for open-ended tasks
Small, unrepresentative test sets
Ignoring regression in existing capabilities
One-time evals without continuous monitoring
Using the same LLM to generate and judge