Summary
P-values give the probability of data under the null; corrections like Bonferroni control false positives across many tests.
In depth
A p-value is not the probability the null is true. Testing many hypotheses inflates false positives: Bonferroni divides alpha by number of tests; Benjamini-Hochberg controls false discovery rate. p-hacking and publication bias are problems.
Pre-registration limits flexibility in analysis. False discovery rate is common in omics. Effect size and confidence interval are more informative than p alone.
Key idea: if you test 20 hypotheses, one is expected to be 'significant' by chance.
A p-value is the probability of obtaining data at least as extreme as observed if the null hypothesis were true. It is not the probability that the hypothesis is true, nor the size or importance of an effect. A threshold of 0.05 is conventional but arbitrary.
When many tests are run, as in genomics with thousands of genes, some will be significant by chance: at p less than 0.05, 5 percent of true nulls give false positives. Corrections control this: Bonferroni divides the threshold by the number of tests, and false discovery rate (Benjamini-Hochberg) controls the expected proportion of false positives. P-hacking (trying analyses until significant) invalidates results.
Example: testing 20,000 genes at 0.05 would give about 1,000 false positives.
Key idea: report effect sizes and confidence intervals as well.
Frequently asked questions
What is P-Values and Multiple Testing?
P-values give the probability of data under the null; corrections like Bonferroni control false positives across many tests.
Which area of biology covers P-Values and Multiple Testing, and how advanced is it?
P-Values and Multiple Testing is one of the 51 topics in Lab Methods & Biostatistics. It is rated Advanced, so it suits learners who want specialist depth. Related topics in this area include Null Hypothesis, Bayesian Statistics, Chi-Squared Test.
Can you give an example of P-Values and Multiple Testing?
Genome-wide studies use p < 5×10^-8.
What should I learn before P-Values and Multiple Testing?
Work through the Beginner topics in Lab Methods & Biostatistics first, such as Scientific Method, Hypothesis, Variables, then return to P-Values and Multiple Testing. Advanced topics assume the Intermediate vocabulary.
What topics are related to P-Values and Multiple Testing?
The closest topics to P-Values and Multiple Testing are Null Hypothesis, Bayesian Statistics, Chi-Squared Test, Statistical Power. Comparing them side by side is a good way to see how the ideas connect.
How can I check that I understand P-Values and Multiple Testing?
Read the summary and the in-depth section, then use the quick check on the interactive lesson for P-Values and Multiple Testing. To practise the whole of Lab Methods & Biostatistics, use the Lab Methods & Biostatistics quiz.
How is P-Values and Multiple Testing different from Null Hypothesis?
P-Values and Multiple Testing: P-values give the probability of data under the null; corrections like Bonferroni control false positives across many tests. Null Hypothesis: The default assumption of no effect or difference, which data may lead us to reject. Read both side by side and note what each one explains.
How should I study P-Values and Multiple Testing step by step?
First, read the summary and write the definition in your own words. Second, read the in-depth section and note any example. Third, compare it with Null Hypothesis and explain the difference aloud. Finally, take the Lab Methods & Biostatistics quiz and come back to this page for any question you miss.