Data scientists: beware of simple metrics

Published: Jan. 5, 2020, 10:54 p.m.

b'Picking a metric for a problem means defining how you\\u2019ll measure success in solving that problem. Which sounds important, because it is, but oftentimes new data scientists only get experience with a few kinds of metrics when they\\u2019re learning and those metrics have real shortcomings when you think about what they tell you, or don\\u2019t, about how well you\\u2019re really solving the underlying problem. This episode takes a step back and says, what are some metrics that are popular with data scientists, why are they popular, and what are their shortcomings when it comes to the real world? There\\u2019s been a lot of great thinking and writing recently on this topic, and we cover a lot of that discussion along with some perspective of our own.\\n\\nRelevant links:\\nhttps://www.fast.ai/2019/09/24/metrics/\\nhttps://arxiv.org/abs/1909.12475\\nhttps://medium.com/shoprunner/evaluating-classification-models-1-ff0730801f17\\nhttps://hbr.org/2019/09/dont-let-metrics-undermine-your-business'