MLOps

Machine Learning Operations: From experimental notebooks to production systems.

What is MLOps?

MLOps (Machine Learning Operations) is a set of practices that combines Machine Learning, DevOps, and Data Engineering to deploy and maintain ML models in production reliably and efficiently. It bridges the gap between data scientists building models and the infrastructure needed to serve them at scale.

💡 Core Principle: MLOps applies DevOps principles (CI/CD, automation, monitoring) to ML systems, addressing unique challenges like data versioning, model drift, and reproducibility.

🔬

Data Science

Experimentation

⚙️

DevOps

Automation

📊

Data Engineering

Pipelines

🔄 The ML Lifecycle

MLOps manages the entire lifecycle of ML models, from development to retirement.

1️⃣

Data Management

Collection, versioning, validation

2️⃣

Model Development

Training, experimentation

3️⃣

Deployment

CI/CD, serving

4️⃣

Monitoring

Performance, drift

🧩 Key Components

📦 Data Versioning

Track changes in datasets over time. Essential for reproducibility and debugging.

DVC Delta Lake LakeFS

🧪 Experiment Tracking

Log hyperparameters, metrics, artifacts, and code versions for every training run.

MLflow Weights & Biases Neptune.ai Comet ML

📋 Model Registry

Central repository for model versions with metadata, lineage, and stage transitions (staging → production).

MLflow Model Registry Vertex AI Model Registry SageMaker Model Registry

🔧 Feature Store

Centralized repository for feature definitions, ensuring consistency between training and serving.

Feast Tecton Vertex Feature Store SageMaker Feature Store

🚀 CI/CD for Machine Learning

ML pipelines require additional CI/CD considerations beyond traditional software.

Pipeline Type	Trigger	Actions
CI (Continuous Integration)	Code push	Lint, unit tests, data validation tests
CT (Continuous Training)	New data, schedule, drift detected	Retrain model, evaluate, register
CD (Continuous Deployment)	New model registered & approved	Canary/Blue-Green deploy, A/B test

# Example: GitHub Actions for ML Pipeline
name: ML Pipeline

on:
  push:
    branches: [main]
  schedule:
    - cron: '0 0 * * 0'  # Weekly retraining

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      
      - name: Install dependencies
        run: pip install -r requirements.txt
      
      - name: Run data validation
        run: python src/validate_data.py
      
      - name: Train model
        run: python src/train.py
        env:
          MLFLOW_TRACKING_URI: ${{ secrets.MLFLOW_URI }}
      
      - name: Evaluate model
        run: python src/evaluate.py
      
      - name: Register model (if passed)
        run: python src/register_model.py

🌐 Model Serving Patterns

Different strategies for serving predictions based on latency, scale, and use case.

🔄 Real-time (Online)

REST/gRPC APIs for instant predictions. Low latency (<100ms).

Fraud detection
Recommendation on click
Chatbots

TensorFlow Serving TorchServe Triton

📦 Batch

Scheduled jobs processing large datasets. Higher throughput.

Daily churn predictions
Monthly risk scores
Batch recommendations

Apache Spark Airflow Dataflow

📱 Edge / Embedded

Models running on devices (mobile, IoT). Offline capable.

Mobile keyboard predictions
Camera apps (face detection)
Smart home devices

TFLite ONNX Runtime Core ML

🌊 Streaming

Process events as they arrive in real-time streams.

IoT sensor anomaly detection
Real-time clickstream analysis
Live event scoring

Apache Kafka Flink Kinesis

📈 Monitoring & Observability

ML systems require monitoring beyond traditional application metrics.

🚨 Model Drift

The phenomenon where model performance degrades over time due to changes in real-world data.

Data Drift

Input distribution changes

Concept Drift

Relationship between X and Y changes

Metric Type	What to Monitor	Tools
Infrastructure	Latency, throughput, CPU/GPU, memory	Prometheus, Grafana, Datadog
Data Quality	Missing values, schema changes, distribution	Great Expectations, Evidently
Model Performance	Accuracy, precision, recall, AUC	MLflow, W&B, Evidently
Business Metrics	Revenue impact, conversion rate, user engagement	Custom dashboards, A/B testing platforms

🎯 Deployment Strategies

🔵🟢 Blue-Green

Run two identical environments. Switch traffic instantly from old (blue) to new (green).

✓ Zero downtime, easy rollback

🐤 Canary

Route a small % of traffic (1-5%) to the new model. Gradually increase if metrics are good.

✓ Low risk, real-world validation

🅰️🅱️ A/B Testing

Split traffic between models and measure business metrics to determine the winner.

✓ Data-driven decisions

📊 MLOps Maturity Levels

Google's MLOps maturity model (from their MLOps whitepaper):

Manual Process

Manual training, manual deployment, Jupyter notebooks

ML Pipeline Automation

Automated training pipeline, manual deployment

CI/CD Pipeline Automation

Fully automated training + deployment, triggered by data/code changes, monitoring

🛠️ Popular MLOps Platforms

Platform	Type	Strengths
Vertex AI	Cloud (GCP)	End-to-end, AutoML, integrated with GCP
SageMaker	Cloud (AWS)	Full lifecycle, Autopilot, AWS ecosystem
Azure ML	Cloud (Azure)	Designer UI, enterprise integration
MLflow	Open Source	Experiment tracking, model registry, flexible
Kubeflow	Open Source (K8s)	Kubernetes-native, scalable pipelines
ZenML	Open Source	Portable pipelines, integrates many tools