Chapter 1.7 — Experiment tracking and model registry¶
🎯 Objective¶
Treat experiments as engineering artifacts.
🧠 Concept¶
- Experiment tracking — recording parameters, code, data, metrics, artifacts and environment.
- Model registry — a versioned catalog of models with metadata, status (staging, production, archived) and approval.
Tools: MLflow, Weights & Biases, DVC, ClearML.
🚨 Failure modes¶
- A "laptop model" with no traceability.
- A metric reported in a deck differs from the reproducible metric.
- A model in production with no traceable dataset/code.
📝 Mandatory templates¶
- Model card — purpose, dataset, metrics, limits, risks.
- Dataset card — origin, schema, bias, restrictions.
- Eval report — versioned comparison between the current model and the candidate.
🧰 Related practical example (planned)¶
EX-ML-03— scikit-learn pipeline + MLflow + local model registry.
📌 Checklist¶
- [ ] Does every experiment have an ID and traceability?
- [ ] Does the production model have a published model card?
- [ ] Does promotion between staging and production require approval?
📚 References¶
- MLflow docs: https://mlflow.org/docs/latest/index.html
- Mitchell et al. — Model Cards for Model Reporting (FAT* 2019): https://arxiv.org/abs/1810.03993
- Gebru et al. — Datasheets for Datasets: https://arxiv.org/abs/1803.09010