Dance p 3 Mar09

Channel:Geoff Cumming · Length:9:36 · Published:2009-03-03 · Views:82841

Watch on YouTube

Summary

Geoff Cumming uses a simulation to show how wildly the p value varies from one replication to the next because of sampling variability. He argues for estimation and confidence intervals over null hypothesis significance testing.

Why HN recommends it

Recommended as a vivid visual demonstration of how unreliable p-values are.

absolutely fantastic and well worth watching — HN, 2015

Data science & statistics · Scientific method & science communication

Subtitles (excerpt)

Psychology and many other disciplines rely on the P value to interpret data. But, can we trust P? You might think of P as the strength of evidence there's an effect. Lower the P, stronger the evidence. There's language to talk about P values. Less than 0.5, significant. Between 0.5 and 1.0, approaching significance or running away from it. This language suggests truth. Three stars, an effect exists. Greater than 1.0, it doesn't. Think how you feel. Three stars, yay. Between 0.5 and 1.0, frustration, so close. And P really matters. Three stars, fame. Greater than 0.5, a waste of effort. Maybe no PhD. So, the P value determines what we believe and what gets published. Does it deserve so much faith? If P reveals truth, a replication should reveal the same truth, so give roughly the same P. Or we could ask, is P a reliable measure of strength of evidence? If so, repeating the experiment should give about the same P. Our basic question, if you repeat an experiment, how much will P vary? If P deserves our trust, the answer needs to be not too much. Let's mark the moods of P values. Three stars, jump for joy, bright red trumpet. Two stars, upbeat clarinet. One star, okay. Between 0.5 and 1.0, question mark, a bit blue, trombone.

On Hacker News

The audio is a bit quiet but the 'dance of the p values' video [1] linked to by the article is absolutely fantastic and well worth watching. [1] [VIDEO] — po, 2015-02
The p-value [1] criterion is often used to test a proposed hypothesis in medical and social science research papers. When you stumble upon something along the lines of "studies that shown that eating broccoli makes people happy" in your everyday life, it comes down to the p-value calculation being small enough <0.05 for the before and after gathered datasets. The p-value method is practically a s… — knowledgesale, 2011-12
I can't claim to understand much of the statistics, but according to [1], p-values are pretty sketchy as a measure of "is this likely to be true?" [1] [VIDEO] — lomnakkus, 2014-03

Related videos