AWS Bedrock - Technical

What is Amazon Bedrock?

Amazon Bedrock is a fully managed service that provides access to leading foundation models from AI companies like Anthropic, Meta, Mistral, and Amazon through a single API. It handles infrastructure, scaling, and security so you can focus on building AI applications.

Multiple Models

One API, many providers

Enterprise Security

IAM, VPC, encryption

AWS Integration

S3, Lambda, SageMaker

Available Foundation Models

Anthropic Claude

Meta Llama

Open Source

Llama 3.2 (1B, 3B, 11B, 90B), Llama 3.1 (8B, 70B, 405B). Open-weight models with strong performance across tasks.

meta.llama3-2-90b-instruct-v1:0 meta.llama3-1-405b-instruct-v1:0

Mistral AI

European AI

Mistral Large, Mistral Small, Mixtral 8x7B. High-quality European models optimized for multilingual tasks.

mistral.mistral-large-2407-v1:0 mistral.mixtral-8x7b-instruct-v0:1

Amazon Titan

AWS Native

Titan Text (Express, Lite, Premier), Titan Embeddings, Titan Image Generator. Amazon's own foundation models optimized for enterprise use.

amazon.titan-text-premier-v1:0 amazon.titan-embed-text-v2:0

Cohere

Enterprise NLP

Command R, Command R+, Embed models. Specialized for RAG, search, and enterprise text understanding.

cohere.command-r-plus-v1:0 cohere.embed-multilingual-v3

Key Features

Guardrails

Built-in content filters, PII detection, topic blocking, and custom policies. Apply safety controls without changing your code.

Knowledge Bases

Managed RAG service. Connect to S3, web crawlers, or databases. Automatic chunking, embedding, and vector search.

Agents

Build autonomous agents that can use tools, execute multi-step tasks, and integrate with AWS Lambda functions.

Fine-Tuning

Customize models with your own data. Continued pre-training and instruction fine-tuning for domain-specific tasks.

Model Evaluation

Compare models on your data with automatic benchmarking. Built-in metrics for accuracy, toxicity, and robustness.

Prompt Flows

Visual workflow builder for chaining prompts, tools, and logic. No-code development for complex AI pipelines.

API Usage

Basic Model Invocation

import boto3
import json

bedrock = boto3.client('bedrock-runtime', region_name='us-east-1')

# Invoke Claude on Bedrock
response = bedrock.invoke_model(
    modelId='anthropic.claude-3-5-sonnet-20241022-v2:0',
    body=json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1024,
        "messages": [
            {"role": "user", "content": "Explain quantum computing simply."}
        ]
    })
)

result = json.loads(response['body'].read())
print(result['content'][0]['text'])

Converse API (Unified Interface)

import boto3

bedrock = boto3.client('bedrock-runtime', region_name='us-east-1')

# Unified API works across all models!
response = bedrock.converse(
    modelId='anthropic.claude-3-5-sonnet-20241022-v2:0',
    messages=[
        {"role": "user", "content": [{"text": "What is machine learning?"}]}
    ],
    inferenceConfig={
        "maxTokens": 1024,
        "temperature": 0.7
    }
)

print(response['output']['message']['content'][0]['text'])

# Same code works with Llama!
response = bedrock.converse(
    modelId='meta.llama3-2-90b-instruct-v1:0',
    messages=[
        {"role": "user", "content": [{"text": "What is machine learning?"}]}
    ],
    inferenceConfig={"maxTokens": 1024}
)

Recommended: Use the Converse API for a unified interface across all models. Switch models without changing your code structure.

Streaming Responses

import boto3

bedrock = boto3.client('bedrock-runtime', region_name='us-east-1')

response = bedrock.converse_stream(
    modelId='anthropic.claude-3-5-sonnet-20241022-v2:0',
    messages=[
        {"role": "user", "content": [{"text": "Write a short story."}]}
    ],
    inferenceConfig={"maxTokens": 2000}
)

# Process stream
for event in response['stream']:
    if 'contentBlockDelta' in event:
        text = event['contentBlockDelta']['delta'].get('text', '')
        print(text, end='', flush=True)

RAG with Knowledge Bases

import boto3

bedrock_agent = boto3.client('bedrock-agent-runtime', region_name='us-east-1')

# Query knowledge base with RAG
response = bedrock_agent.retrieve_and_generate(
    input={"text": "What are our return policies?"},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": "KB-12345",
            "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
        }
    }
)

print(response['output']['text'])

# Access source citations
for citation in response.get('citations', []):
    print(f"Source: {citation['retrievedReferences'][0]['location']}")

Using Guardrails

import boto3

bedrock = boto3.client('bedrock-runtime', region_name='us-east-1')

# Apply guardrails to any model invocation
response = bedrock.converse(
    modelId='anthropic.claude-3-5-sonnet-20241022-v2:0',
    messages=[
        {"role": "user", "content": [{"text": "Tell me about our competitors."}]}
    ],
    guardrailConfig={
        "guardrailIdentifier": "my-guardrail-id",
        "guardrailVersion": "DRAFT"
    }
)

# Check if guardrail was triggered
if response.get('stopReason') == 'guardrail_intervened':
    print("Content blocked by guardrail")
else:
    print(response['output']['message']['content'][0]['text'])

Pricing (On-Demand)

Model	Input (1M tokens)	Output (1M tokens)	Context
Claude 3.5 Sonnet	$3.00	$15.00	200K
Claude 3.5 Haiku	$0.80	$4.00	200K
Llama 3.2 90B	$0.72	$0.72	128K
Mistral Large	$4.00	$12.00	128K
Titan Text Premier	$0.50	$1.50	32K

Provisioned Throughput: For predictable workloads, purchase model units for consistent performance and potential cost savings.

Best Practices

Use Converse API

The Converse API provides a unified interface across all models. Makes it easy to switch providers without code changes.

VPC Endpoints

Use VPC endpoints for private connectivity. Keep traffic within AWS network without traversing the internet.

Enable Guardrails

Configure guardrails for PII detection, content filtering, and topic blocking. Essential for production applications.

Monitor with CloudWatch

Enable CloudWatch metrics and Model Invocation Logging. Track latency, errors, and token usage for optimization.

Amazon Bedrock