DevOps to MLOps Transition

MLOps extends DevOps principles to machine learning by treating data, models, and pipelines as first-class versioned artifacts. This topic covers the operational differences, tooling, team roles, and CI/CD adaptations needed to reliably ship and maintain ML systems in production.

Overview

Why DevOps alone isn't enough for ML

DevOps was built around one artifact, code: the same source always builds the same binary. A trained model is the product of code, data and hyperparameters, and any of them can change its behavior without a line of source changing.

Production ML also fails in ways DevOps never anticipated: data drift, concept drift, training-serving skew and stale feature distributions. A model that was 94% accurate at launch can drop to 78% six months later because the world changed, not the code.

> [!NOTE] MLOps applies DevOps automation, reproducibility and observability to the full ML lifecycle, and adds the data and model dimensions that software CI/CD ignores.

The three pillars

When one pillar is weak the whole system degrades. A great model on bad data pipelines fails in production; a well-trained model with no monitoring decays silently.

CI/CD for ML: two loops

Traditional CI/CD runs lint → unit test → build → integration test → deploy. ML CI/CD runs two nested loops.

Inner — Locally or in notebooks — Data scientists, iteratively — Candidate models
Outer — Automated pipeline — Code commits, data changes, schedules — Promoted, monitored models

What ML adds over traditional CI/CD

Data validation: Great Expectations or Pandera check schema, distribution and freshness.
Feature pipeline tests: no leakage, correct join keys, temporal consistency.
Model evaluation gates: accuracy, fairness, latency and calibration thresholds.
Model registry: staged promotion (staging → production) and rollback.
Drift detection: production monitoring that can trigger retraining.

> [!WARNING] A model that passes unit tests but fails statistical evaluation on a holdout set must never reach the registry.

Roles and team topology

Data Scientist — Problem framing, feature selection, architecture, offline evaluation — Inner loop, via MLflow and the feature store
ML Engineer — Productionized feature pipelines, inference optimization, CI training code, serving layer — The bridge between both loops
MLOps / Platform Engineer — Kubeflow or Vertex AI pipelines, Feast, registry, monitoring, CI/CD — The platform; a DevOps engineer who is ML-aware
Data Engineer — Data pipelines, data quality SLAs, catalog metadata, data contracts — Upstream of feature engineering

At small companies these roles collapse into fewer people; at hyperscalers they split further, for example into a dedicated ML Infrastructure team.

Tool landscape

Experiment tracking — MLflow, Weights and Biases, Neptune
Pipeline orchestration — Kubeflow Pipelines, Vertex AI Pipelines, Airflow with ML operators, Prefect, ZenML
Feature stores — Feast, Tecton, Vertex AI Feature Store, Azure ML Feature Store
Model registry — MLflow Model Registry, Azure ML, SageMaker, Hugging Face Hub
Model serving — KServe, Triton Inference Server, BentoML, Seldon Core, Azure ML Online Endpoints
Data versioning — DVC, LakeFS, Delta Lake
Monitoring — Evidently AI, Arize, WhyLabs, Fiddler
Infrastructure — Azure ML CLI v2, SageMaker Pipelines SDK, Vertex AI SDK, Terraform

Security, compliance and model maintenance

Model lineage — Regulators in finance and healthcare expect you to rebuild any model version from its exact data snapshot, code commit and hyperparameters: immutable dataset versions plus a registry with lineage metadata
Data governance — Audited access to PII in training data; models may need retraining or deletion when a user invokes right to erasure (GDPR Article 17)
Model decay — Ongoing maintenance even with zero code changes: scheduled retraining, drift thresholds, champion/challenger testing

> [!IMPORTANT] Unlike software, a deployed model needs maintenance even when nobody touches the code.

Interview questions

Explain the key differences between traditional CI/CD and ML CI/CD pipelines. What stages does ML add and why?
What are the MLOps maturity levels and how do they guide an organization's transition from DevOps?
How does model decay differ from software bugs, and what operational practices does MLOps use to address it?
Compare the roles of ML Engineer, Data Scientist, and MLOps Engineer in a production ML team. Where do responsibilities overlap and where do they conflict?

More MLOps & LLMOps topics

Start free on Cariara · Coding problems · System design studio