Interactive Simulation

Why Multiple Comparisons
Are Dangerous

Each test has a 5% false positive rate. What happens when you run many tests on data where nothing is real?

0
"Significant"
0
Not significant
0%
False positive rate
โ€”
P(at least 1 FP)
๐Ÿงช

Individual Test Results

๐Ÿ“Š

Running Tally

Significant
0
Not sig.
0
How it works: We simulate running multiple independent statistical tests where the null hypothesis is true every time โ€” there is no real effect.

Each test still has a α = 5% chance of a false positive. With n tests, the probability of at least one false positive is:
P = 1 − (1 − α)n