What a p-value is, precisely
A p-value is the probability of observing a test statistic at least as extreme as the one you got, assuming the null hypothesis is true. That conditional clause is the whole definition, and dropping it produces most of the misinterpretations in circulation.
It is not the probability that the null hypothesis is true. It is not the probability that your result was a fluke. It is not one minus the probability that the alternative is true. It is a statement about how unusual your data would be in a world where the effect is exactly zero — nothing more, and that world is a hypothesis, not a fact.
The p-value also does not measure the size of an effect. A tiny difference measured on a million observations produces a minuscule p-value; a large difference measured on eight observations may produce a large one. The statistic that goes into this calculator already blends effect size with sample size, and the p-value inherits that blend. Always read it alongside a confidence interval or an effect size such as Cohen's d.
What it does do well is make different tests comparable. A t of 2.23 on 10 degrees of freedom and a χ² of 3.84 on 1 degree of freedom are not comparable numbers; both have a p-value of 0.05, and that is.
How the tail area is computed
Every p-value is an area under a probability density curve. The reference distribution is fixed by your test, the observed statistic marks a point on the horizontal axis, and the p-value is the area beyond that point in whichever direction your alternative hypothesis allows.
For a right-tailed alternative the p-value is 1 − F(x), where F is the cumulative distribution function — the area to the left. For a left-tailed alternative it is simply F(x). For a two-tailed alternative you take the area beyond |x| and double it, because a result equally extreme in the other direction would have counted as evidence too.
Which distribution you use depends on the test. The standard normal serves large-sample tests and proportion tests, and it takes no degrees of freedom. Student's t serves means and regression coefficients, with n − 1 or n − k degrees of freedom, and has heavier tails than the normal at small df. Chi-square serves contingency tables and goodness-of-fit tests; it is a sum of squared standardised deviations, so it is never negative and only its upper tail is used. F serves analysis of variance and model comparisons; it is a ratio of two variance estimates, carries two separate df parameters, and again uses only the upper tail.
These distributions are related. Chi-square with 1 degree of freedom is exactly the square of a standard normal, which is why χ² = 3.841459 and z = ±1.959964 both give p = 0.05. F with 1 numerator degree of freedom is the square of the corresponding t, which is why a two-group ANOVA and a two-sample t-test always agree.
Worked example: three statistics, three routes to the same p
Case 1: a t statistic. A regression output reports a coefficient with t = 2.2281 on 10 degrees of freedom, two-tailed. The area of the t(10) distribution above 2.2281 is 0.025. Double it: p = 0.0500. This is exactly the published two-tailed 5% critical value for 10 df, so the result sits precisely on the conventional threshold.
Case 2: the same statistic, one-tailed. If you had pre-registered the direction, the p-value is the single tail area, 0.025 — exactly half the two-tailed value. That halving is why choosing the tail after seeing the data is illegitimate: it converts a p of 0.05 into 0.025 with no new evidence at all.
Case 3: a chi-square. A 2×2 test of independence gives χ² = 6.635 with 1 degree of freedom. The upper tail area is 0.010, with no doubling, because the chi-square statistic already aggregates deviations in both directions. Check the relationship: √6.635 = 2.576, which is the two-tailed 1% z cut-off. The two tests are the same test wearing different clothes.
Reporting all three: “t(10) = 2.23, p = 0.050”, “t(10) = 2.23, one-tailed p = 0.025”, “χ²(1) = 6.64, p = 0.010”. Always give the statistic, the degrees of freedom and the p-value together; a bare p-value cannot be checked by anyone.
Reading the number you get
Compare it against an alpha you chose in advance. Alpha is the false-positive rate you accept: at 0.05, one in twenty tests of a true null will reject by chance. That is a property of the procedure, not of your particular study, and it only holds if alpha was fixed before the data spoke.
Report the exact value, not a threshold. “p = 0.043” is more informative than “p < 0.05”, and far more informative than “p = n.s.”. Below about 0.0001 the exact figure stops meaning much, because it depends on distributional assumptions in a region where you have no data; “p < 0.0001” is the honest form there.
Do not read a large p-value as evidence of no effect. It means your data are compatible with the null, which is also true of a study too small to detect anything. The distinction between “no effect” and “no power” lives in the confidence interval, not the p-value — see the statistical power calculator.
Count how many tests you ran. Twenty independent tests of true nulls will, on average, produce one p-value below 0.05. If you tested many outcomes, subgroups or model specifications and are reporting the one that worked, the nominal p-value overstates the evidence badly and needs a multiplicity correction.
Normal (z) statistics and their p-values
| z | One-tailed p | Two-tailed p |
|---|---|---|
| 1.000 | 0.1587 | 0.3173 |
| 1.500 | 0.0668 | 0.1336 |
| 1.645 | 0.0500 | 0.1000 |
| 1.960 | 0.0250 | 0.0500 |
| 2.000 | 0.0228 | 0.0455 |
| 2.326 | 0.0100 | 0.0200 |
| 2.576 | 0.0050 | 0.0100 |
| 3.000 | 0.0013 | 0.0027 |
| 3.291 | 0.0005 | 0.0010 |
Standard published normal tail areas. For t statistics the p-value is larger at the same statistic value, converging on these figures as the degrees of freedom grow.
How p-values get misused
- Choosing one-tailed after seeing the direction. This halves the p-value without adding evidence and doubles the real false-positive rate.
- Treating 0.049 and 0.051 as different in kind. Nothing changes discontinuously at 0.05; the threshold is a convention, and results near it should be reported with the exact value and an effect size.
- Reading p as the probability the null is true. It is computed assuming the null is true, so it cannot also be a probability about it.
- Testing many things and reporting the winner. Twenty tests of true nulls yield about one p below 0.05 by construction. Correct for multiplicity or declare the primary outcome in advance.
- Using the wrong degrees of freedom. A t of 2.228 sits exactly on the two-tailed 5% threshold at 10 df, but that threshold is 2.571 at 5 df and 2.086 at 20 df — so the identical statistic is significant on 20 df and not significant on 5. Copy df from your output rather than reconstructing it.
- Doubling a chi-square or F p-value. Those tests are inherently one-sided; the statistic already sums deviations in every direction.
Getting the statistic in the first place
This calculator converts a statistic you already have. To produce one, use the test that matches your design: the one-sample t-test for a mean against a reference value, the two-sample t-test for two independent groups, the one-way ANOVA for three or more, the chi-square test of independence for a contingency table, and the two-proportion z-test for comparing two rates. Each of those reports its own p-value; this page is for statistics that arrive from software, a paper or a textbook exercise without one.
Going the other way — from a probability to a cut-off — is what the critical value calculators do: z, t and chi-square. Critical values and p-values are two views of the same tail area, and the two comparisons always agree when they use the same alpha and the same number of tails.
The American Statistical Association's 2016 statement on p-values sets out six principles that are worth reading in full; the shortest summary is that a p-value is one piece of evidence about one hypothesis, and no scientific conclusion should rest on whether it crosses 0.05.
Key terms
- Tail area
- The probability that the reference distribution exceeds the observed statistic, in the direction the alternative allows.
- Critical value
- The statistic whose tail area equals alpha. Rejecting when |statistic| exceeds it is equivalent to rejecting when p < α.
- Degrees of freedom
- The parameter that fixes the shape of the t, chi-square and F distributions. Two statistics of equal magnitude give different p-values on different df.
- Alpha
- The false-positive rate accepted in advance. It sets the critical value but never changes the p-value.