11-07, 15:20–16:00 (US/Eastern), Radio City
"Why Your Machine Learning Model's Probabilities Are Lying to You: A Guide to Probability Calibration" is a deep dive into the often-overlooked aspect of machine learning models—probability calibration. This talk will reveal why the probabilities produced by your models may be misleading, how this can impact decision-making, and what you can do to fix it. Attendees will leave with a solid understanding of probability calibration techniques and their practical applications in real-world scenarios.
In this talk, I will explore the importance of probability calibration in machine learning models, a critical yet frequently neglected aspect of model evaluation. Probability calibration ensures that the predicted probabilities of your models align with the actual likelihood of events, making your models more reliable for decision-making processes.
Target Audience: This talk is tailored for data scientists, machine learning practitioners, and engineers who work with predictive models and are interested in improving the trustworthiness of their model outputs.
Structure:
Introduction (0-5 mins): Overview of probability calibration and why it matters.
Understanding the Problem (5-10 mins): Common issues with uncalibrated probabilities and their impact on real-world applications.
Methods for Calibration (10-20 mins): Detailed walkthrough of popular calibration techniques such as Platt Scaling & Isotonic Regression, including their pros and cons.
Practical Applications (20-25 mins): Case studies demonstrating the impact of probability calibration on decision-making in various domains.
Q&A (25-30 mins): Open floor for questions and discussion.
Takeaway: By the end of the session, attendees will have a clear understanding of how to assess and improve the calibration of their machine learning models, leading to more accurate and reliable predictions. No advanced mathematical background is required, but a basic understanding of machine learning concepts is recommended.
Previous knowledge expected
My name is Allan Butler. I do/am a Data Scientist at H-E-B. Previously I was a data scientist in the energy sector as well as the professional sports world where I worked on a wide variety of problems related to demand forecasting, pricing optimization, and visual story telling.