Telco Churn MLOps Pipeline
LiveOverview
This project implements a production-grade MLOps pipeline for predicting telecom customer churn. It covers data ingestion, preprocessing, model training, experiment tracking, workflow orchestration, streaming inference, and REST API deployment.
The pipeline is the final stage of a four-phase development series:
- Phase 1: EDA, data exploration, and preprocessing
- Phase 2: Model training, evaluation, and hyperparameter tuning
- Phase 3: Production pipeline architecture with monitoring
- Phase 4: Full MLOps integration - Kafka, Airflow, MLflow, PySpark, Docker, and API deployment
Architecture
Challenges & Solutions
Churn prediction has asymmetric business cost - missing a churner is far more expensive than targeting an extra customer.
Optimized the classification threshold for recall over accuracy, ensuring the model catches more real churners even at a small precision cost.
Most notebook ML projects don't prove production readiness to engineering teams.
Added Docker, Airflow, Kafka, MLflow, pytest, configuration management, and a REST API to demonstrate full production deployment capability.
Real-time inference needs decoupled, fault-tolerant data flow that can handle variable input rates.
Implemented a Kafka producer/consumer pipeline that decouples data ingestion from inference and delivers 8.2ms streaming prediction latency.
Key Metrics
ROC-AUC
84.66%
GradientBoosting recall-optimized
Recall
80.75%
Optimized for churn detection
Inference Latency
8.2ms
Streaming inference via Kafka
Test Coverage
96.4%
212/220 tests passing
Features
- Automated CSV ingestion, validation, and feature engineering (19 → 45 features)
- scikit-learn GradientBoosting with recall-focused threshold optimization
- MLflow experiment tracking and model registry with 15+ model versions
- Kafka producer/consumer for real-time streaming inference at 8.2ms latency
- Airflow DAGs for batch pipeline, streaming triggers, and validation workflows
- Dockerized Flask REST API with prediction and health endpoints
- PySpark distributed training path for scale-out workloads
- Data quality checks, schema validation, and configuration management
