GenAIHub
EN | PT
← Back to Technical Section

Prompt Engineering

The Art and Science of Communicating with Large Language Models

What is Prompt Engineering?

Prompt engineering is the practice of designing and optimizing inputs (prompts) to elicit desired outputs from Large Language Models (LLMs). It's a critical skill for building reliable AI-powered applications, as the quality of the prompt directly impacts the quality, consistency, and safety of model responses.

Unlike traditional programming where you write explicit instructions, prompt engineering requires understanding how models interpret natural language, leverage context, and generalize from examples.

Core Prompting Strategies

Zero-shot Prompting

Ask the model to perform a task without providing any examples. Relies entirely on the model's pretrained knowledge and instruction-following capabilities.

Zero-shot Example
Classify the following text as positive, negative, or neutral:

Text: "The new update completely broke my workflow. Very disappointed."

Classification:

Best for: Simple tasks, well-defined categories, strong base models (GPT-4, Claude 3)

Few-shot Prompting

Provide 2-5 examples demonstrating the desired input-output pattern before asking the model to handle a new case. This leverages in-context learning—one of the most powerful capabilities of modern LLMs.

Few-shot Example
Extract the company name and valuation from the text.

Text: "Stripe has raised $600M at a $95B valuation."
Output: {"company": "Stripe", "valuation": "$95B"}

Text: "OpenAI is now valued at $80 billion after new investment."
Output: {"company": "OpenAI", "valuation": "$80B"}

Text: "Anthropic closed a $2B round, pushing its valuation to $18B."
Output:

Best for: Structured extraction, classification with custom categories, format enforcement

Chain-of-Thought (CoT) Prompting

Encourage the model to "think step by step" before providing the final answer. This dramatically improves performance on reasoning, math, and multi-step problems.

Chain-of-Thought Example
Q: A store has 23 apples. They sell 17 and receive a shipment of 31 more.
   How many apples do they have now?

A: Let me think step by step:
   1. Starting apples: 23
   2. After selling 17: 23 - 17 = 6
   3. After receiving 31: 6 + 31 = 37
   
   The store has 37 apples.

Trigger phrases: "Let's think step by step", "Think through this carefully", "Break this down into steps"

Zero-shot Chain-of-Thought

Simply adding "Let's think step by step" to a zero-shot prompt can significantly improve reasoning performance without needing examples.

Zero-shot CoT
Q: If it takes 5 machines 5 minutes to make 5 widgets, 
   how long would it take 100 machines to make 100 widgets?

Let's think step by step.

Advanced Prompting Techniques

Self-Consistency

Generate multiple reasoning paths (with temperature > 0) and take the majority vote. Improves accuracy on complex reasoning tasks by 5-20%.

Tree of Thoughts (ToT)

Explore multiple reasoning branches, evaluate each, and backtrack when needed. Enables deliberate problem-solving for complex tasks.

ReAct (Reason + Act)

Interleave reasoning traces with action steps (tool calls, searches). Foundation for agentic systems and tool-using LLMs.

Reflexion

Have the model critique its own output and iteratively improve. Useful for self-correction and quality improvement.

Retrieval-Augmented Prompting

Inject relevant retrieved context into the prompt. Grounds responses in factual, up-to-date information.

Structured Output Prompting

Force JSON, XML, or other structured formats using schema definitions and explicit format instructions.

Anatomy of an Effective Prompt

A well-structured prompt typically contains these components:

Prompt Structure Template
# ROLE / PERSONA
You are an expert data analyst specializing in financial metrics.

# CONTEXT
You are analyzing quarterly earnings reports for Fortune 500 companies.
The user will provide excerpts from 10-K filings.

# TASK
Extract the following metrics from the provided text:
- Revenue (in millions USD)
- Net Income (in millions USD)
- Year-over-Year Growth (%)

# FORMAT
Return your response as a JSON object with this structure:
{
  "revenue_m": number,
  "net_income_m": number,
  "yoy_growth_pct": number,
  "confidence": "high" | "medium" | "low"
}

# CONSTRAINTS
- If a metric is not found, use null
- Do not make up numbers; only extract explicitly stated values
- Include a confidence level based on data clarity

# EXAMPLES (optional)
Input: "Total revenue reached $45.2 billion, up 12% from last year..."
Output: {"revenue_m": 45200, "net_income_m": null, "yoy_growth_pct": 12, "confidence": "medium"}

# INPUT

System Prompts

System prompts (supported by OpenAI, Anthropic, and most APIs) set persistent context and behavioral guidelines that apply across the entire conversation.

System Prompt Example
You are a senior software engineer at a fintech company.

Guidelines:
- Always prioritize security and data privacy in your recommendations
- Use Python for code examples unless the user specifies otherwise
- When discussing architecture, consider scalability and cost
- Be concise but thorough; avoid unnecessary disclaimers
- If you're uncertain, say so rather than guessing

Never:
- Suggest storing sensitive data in plain text
- Recommend deprecated libraries or APIs
- Provide medical, legal, or financial advice

System Prompt Best Practices

  • Define a clear persona and expertise level
  • Specify what the model should and should NOT do
  • Set output format preferences (concise, detailed, technical level)
  • Include domain-specific terminology or rules
  • Add safety guardrails for sensitive applications

Common Pitfalls & How to Avoid Them

Pitfall Problem Solution
Vague Instructions "Summarize this" → inconsistent length/style "Summarize in 2-3 sentences for a technical audience"
Missing Format Spec Model returns prose when you need JSON Explicitly define output schema with examples
Negative Framing "Don't be verbose" → model focuses on "verbose" "Be concise and direct"
Context Overload Too much context → model ignores key parts Prioritize relevant info; use chunking for long docs
No Error Handling Model hallucinates when data is missing "If not found, return null" or "Say 'I don't know'"
Prompt Injection Risk User input overrides system instructions Clearly delimit user input; add injection defenses

Prompt Optimization Workflow

  1. Start Simple: Write a basic zero-shot prompt and test on 10-20 examples
  2. Analyze Failures: Categorize errors (wrong format, hallucination, misunderstanding)
  3. Add Specificity: Add constraints, format specs, or examples to address failures
  4. Try CoT: For reasoning tasks, add "think step by step"
  5. Add Few-shot Examples: Include 2-5 diverse examples covering edge cases
  6. Evaluate at Scale: Test on 100+ examples; measure accuracy, latency, cost
  7. A/B Test: Compare prompt variants in production
  8. Document & Version: Track prompt versions with their performance metrics

💡 Pro Tip: Treat prompts like code—use version control, write tests, and establish review processes for production prompts.

Key Model Parameters

Parameter Description Typical Values
temperature Controls randomness. Lower = more deterministic 0.0 (exact), 0.7 (balanced), 1.0+ (creative)
top_p Nucleus sampling. Considers tokens within cumulative probability p 0.9-0.95 for most tasks
max_tokens Maximum length of generated response Depends on task; 256-4096 common
stop Sequences that halt generation ["\n\n", "###", ""]
frequency_penalty Reduces repetition of tokens already used 0.0-0.5

When to Use Which Temperature

  • temperature=0: Code generation, data extraction, classification
  • temperature=0.3-0.5: Technical writing, summarization
  • temperature=0.7-0.9: Creative writing, brainstorming
  • temperature=1.0+: Poetry, highly creative tasks (use with caution)

Tools for Prompt Engineering

  • LangSmith: Prompt testing, tracing, and evaluation platform
  • Promptfoo: Open-source prompt testing and comparison tool
  • OpenAI Playground: Interactive prompt testing with parameter controls
  • Anthropic Workbench: Claude-specific prompt development environment
  • PromptLayer: Prompt versioning and analytics
  • Weights & Biases Prompts: Experiment tracking for prompts

Related Topics

Test Your Knowledge

Score 8/10 or higher to pass