#188 - AMA #30: How to Read and Understand Scientific Studies

Published: Dec. 20, 2021, 9 a.m.

In this \u201cAsk Me Anything\u201d (AMA) episode, Peter and Bob dive deep into all things related to studying studies to help one sift through all the noise to find the signal. They define the various types of studies, how a study progresses from idea to execution, and how to identify study strengths and limitations. They explain how clinical trials work, as well as the potential for bias and common pitfalls to watch out for. They dig into key factors that contribute to the rigor (or lack thereof) of an experiment, and they discuss how to measure effect size, differentiate relative risk from absolute risk, and what it really means when a study is statistically significant. Finally, Peter lays out his personal process when reading through scientific papers.


If you\u2019re not a subscriber and listening on a podcast player, you\u2019ll only be able to hear a preview of the AMA. If you\u2019re a subscriber, you can now listen to this full episode on your\xa0private RSS feed\xa0or on our website at the\xa0AMA #30 show notes page. If you are not a subscriber, you can learn more about the subscriber benefits\xa0here.


We discuss:

  • The ever changing landscape of scientific literature [2:15];
  • The process for a study to\xa0progress\xa0from idea to design to execution [4:15];
  • The various types of studies and how they differ [7:30];
  • The different phases of a clinical trial [19:15];
  • Observational studies and the potential for bias [26:30];
  • Experimental studies: Randomization, blinding, and other factors that make or break a study [44:00];
  • Power, p-values, and statistical significance [56:15];
  • Measuring effect size: Relative risk vs. absolute risk, hazard ratios, and \u201cNumber Needed to Treat\u201d [1:07:45];
  • How to interpret confidence intervals [1:17:30];
  • Why a study might\xa0be stopped before its completion\xa0[1:23:45];
  • Why only\xa0a fraction of\xa0studies\xa0are ever\xa0published and how to combat publication bias [1:31:30];
  • Why certain journals are more respected than others [1:40:30];
  • Peter\u2019s process when reading a scientific paper [1:43:45]; and
  • More.