Keeping ourselves honest when we work with observational healthcare data

Published: April 20, 2020, 2:43 a.m.

b'The abundance of data in healthcare, and the value we could capture from structuring and analyzing that data, is a huge opportunity. It also presents huge challenges. One of the biggest challenges is how, exactly, to do that structuring and analysis\\u2014data scientists working with this data have hundreds or thousands of small, and sometimes large, decisions to make in their day-to-day analysis work. What data should they include in their studies? What method should they use to analyze it? What hyperparameter settings should they explore, and how should they pick a value for their hyperparameters? The thing that\\u2019s really difficult here is that, depending on which path they choose among many reasonable options, a data scientist can get really different answers to the underlying question, which makes you wonder how to conclude anything with certainty at all.\\n\\nThe paper for this week\\u2019s episode performs a systematic study of many, many different permutations of the questions above on a set of benchmark datasets where the \\u201cright\\u201d answers are known. Which strategies are most likely to yield the \\u201cright\\u201d answers? That\\u2019s the whole topic of discussion.\\n\\nRelevant links:\\nhttps://hdsr.mitpress.mit.edu/pub/fxz7kr65'