Is Most Published Research Wrong?
Channel:Veritasium · Length:12:22 · Published:2016-08-11 · Views:6205301
Summary
Veritasium explains why a large share of published research findings may be false, covering p-values, p-hacking, publication bias and low replication rates. It draws on John Ioannidis's well-known paper and describes reforms such as preregistration.
Why HN recommends it
HN commenters link it whenever a shaky study or the p<0.05 threshold comes up, as a short primer on false positives in science.
This video is a good primer — HN, 2018
Data science & statistics · Scientific method & science communication
Subtitles (excerpt)
In 2011 an article was published in the reputable "Journal of Personality and Social Psychology". It was called "Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect" or, in other words, proof that people can see into the future. The paper reported on nine experiments. In one, participants were shown two curtains on a computer screen and asked to predict which one had an image behind it, the other just covered a blank wall. Once the participant made their selection the computer randomly positioned an image behind one of the curtains, then the selected curtain was pulled back to show either the image or the blank wall the images were randomly selected from one of three categories: neutral, negative, or erotic. If participants selected the curtain covering the image this was considered a hit. Now with there being two curtains and the images positions randomly behind one of them, you would expect the hit rate to be about fifty percent. And that is exactly what the researchers found, at least for negative neutral images however for erotic images the hit rate was fifty-three percent. Does that mean that we can see into the future? Is that slight deviation significant? Well to
On Hacker News
Wikipedia is infamously bad at teaching math. This Veritasium video does a great job at explaining how such skewed priors can easily appear in our current academic system and the paradox in general: [VIDEO] — KK7NIL, 2025-12
Please watch this video. It neatly shows why should you treat small and less rigorous studies with suspicion, even if there are many giving some results: [VIDEO] — scotty79, 2021-08
Cue “p = 0.05 is a random choice” and a big thread about p-hacking. Any takers? It feels like everything to say has already been said about the bias of small sample sizes and the dangers of p-hacking. But it also seems like the discussion is inevitable here. I wonder if there’s a new insight to glean, somehow... For a background on why p-values are misleading, see [VIDEO] which is excellent. He p… — sillysaurusx, 2020-10
If the test data does not have a 50/50 split (or something around that ratio), the headline is straight up lies. Skewed test data is the most common problem with research, reminds me of this great video by Veritasium (Is most published research wrong?): [VIDEO] — r_singh, 2018-02
My dude. Please go get yourself some education on how science works. I feel embarrassed for you. The reproducibility crisis is about the prevalence of false positive results in the scientific literature. This video is a good primer: [VIDEO] A false positive is when a paper says "this works" when it actually doesn't, because of a statistical fluke in their sample. Even if citing the reproducibilit… — hannasanarion, 2018-11
Related videos
- that dark matter video aged like milk · Angela Collier · 1:00:28
- Revolutionary Paint: How to Make Surfaces Stay Cool in the Sun · Tech Ingredients · 49:39
- Monads and Gonads · Google TechTalks · 49:47
- Dr Russell Barkley on ADHD Meds and how they all work differently from each other · Adhd Videos · 5:50
- This is how you treat ADHD based off science, Dr Russell Barkley part of 2012 Burnett Lecture · Adhd Videos · 13:46
- The Bayesian Trap · Veritasium · 10:36
- Training AI Without Writing A Reward Function, with Reward Modelling · Robert Miles AI Safety · 17:51
- The Battle of Midway 1942: Told from the Japanese Perspective (1/3) · Montemayor · 41:46
- Why Meat is the Best Worst Thing in the World 🍔 · Kurzgesagt – In a Nutshell · 7:44
- Ben Marking's Fake BBC Nuclear Attack Broadcast (Full Resolution, Uncut, Unedited) · Tooth · 55:42
- The Zipf Mystery · Vsauce · 21:05
- Antonin Scalia - On American Exceptionalism · AmericanRhetoric.com · 7:21
- Self: The Movie; · Jecel Assumpção Jr · 21:01
- 30 Weird Chess Algorithms: Elo World · suckerpinch · 42:35
- 4.5 Billion Years in 1 Hour · Kurzgesagt – In a Nutshell · 1:03:31
- How does a whip break the sound barrier? (Slow Motion Shockwave formation) - Smarter Every Day 207 · SmarterEveryDay · 11:21
- The Constitution of Athens · Historia Civilis · 19:20