What a one-sample t-test decides
You have one group of measurements and one reference number. A machine is supposed to fill bottles to 500 mL; you measure 25 and get 520. A drug's label claims a mean onset of 30 minutes; your trial averages 34. A supplier promises 500 MPa; your sample averages 520. In every case the question is the same: is the gap between your average and the reference bigger than sampling noise can comfortably explain?
The null hypothesis H₀ says the population mean equals μ₀ and the gap you saw is just the luck of which items you happened to measure. The test computes how surprising your data would be if that were true. The answer is the p-value: the probability of seeing a difference at least this extreme, in the direction your alternative allows, when H₀ is exactly true.
Two things about the p-value cause most of the confusion in applied work. It is not the probability that H₀ is true — it is a probability computed assuming H₀ is true. And a large p-value is not evidence that H₀ is true; it means your data cannot distinguish μ₀ from the alternatives, which with a small sample is a statement about your study rather than about the world. That is why this calculator always reports a confidence interval for the difference alongside the p-value: the interval shows exactly which differences remain compatible with your data.
Why the statistic has this shape
The t statistic is a signal-to-noise ratio. The signal is x̄ − μ₀, the raw distance between what you measured and what H₀ claims. The noise is the standard error, s/√n, which is how far a sample mean of that size typically strays from the true mean. Divide one by the other and you get the distance in units of its own typical size, which is what makes t comparable across studies with different units.
Two levers move t. A bigger difference moves it directly and proportionally. A bigger sample moves it as √n: quadruple the observations and the same raw difference produces twice the t. That second lever is why a trivial difference can be highly significant in a large study, and why an important difference can fail to reach significance in a small one. Statistical significance and practical importance are different questions, and only the effect size answers the second — Cohen's d, the difference divided by the standard deviation rather than by the standard error, is reported in the steps above for that reason.
The reference distribution is Student's t with n − 1 degrees of freedom rather than the normal, because you estimated the noise from the same small sample that produced the signal. That estimate is itself uncertain, and the t distribution's heavier tails are the price. If you somehow knew the population standard deviation, you would compare against the normal instead, and the critical value at 95% would be 1.960 regardless of n. You can look either value up with the t critical value calculator or the z critical value calculator.
Worked example: 25 components against a 500 MPa specification
A supplier's components are specified at 500 MPa mean tensile strength. You test 25 units, obtaining x̄ = 520 MPa and s = 50 MPa. You had no prior direction in mind, so the alternative is two-tailed at α = 0.05.
- Standard error. SE = 50/√25 = 50/5 = 10.000 MPa.
- Difference. x̄ − μ₀ = 520 − 500 = 20 MPa.
- t statistic. t = 20 / 10 = 2.000.
- Degrees of freedom. df = 25 − 1 = 24.
- Critical value. Two-tailed at α = 0.05 with df = 24, the table gives 2.0639.
- Compare. |t| = 2.000 < 2.0639, so you fail to reject H₀. The exact two-tailed p-value is 0.0567.
- Confidence interval for the difference. 20 ± 2.0639 × 10 = 20 ± 20.639, so −0.64 to 40.64 MPa.
Read the interval and the p-value together. The interval includes zero — which is exactly why the test does not reject — but only barely, and it also includes differences as large as 40 MPa. “Not significant” here means “this study cannot tell a real 20 MPa advantage from noise”, not “there is no advantage”. Cohen's d = 20/50 = 0.4, a moderate effect by conventional labels.
Now suppose you had tested 40 components with the same mean and standard deviation. SE becomes 50/√40 = 7.906, t becomes 2.530, df = 39, the two-tailed critical value is 2.023, and you would reject. Nothing about the components changed; only your ability to see the difference did.
Reading the output
Start with the confidence interval, not the verdict. The interval for the difference tells you the size of effect your data support. If the whole interval lies inside a range you would call unimportant, you have a genuinely negative result. If the interval is wide and spans both trivial and large effects, you have an inconclusive study whatever the p-value says.
Compare the p-value against the alpha you fixed before looking at the data. Alpha is your tolerance for a false positive: at α = 0.05, one in twenty tests of a true null will reject by chance. Choosing alpha after seeing the p-value, or running the test both ways and reporting the tail that worked, inflates that error rate in ways the number no longer reflects.
Check the direction if you ran a one-tailed test. A one-tailed test at α = 0.05 uses the 1.7109 cutoff at df = 24 rather than 2.0639, so it detects a difference in the nominated direction more easily. The price is absolute: a difference in the other direction cannot be declared significant no matter how large it is. Choose one-tailed only when a result in the opposite direction would lead to the same action as no result at all.
Watch the assumptions. The test needs independent observations, and it needs the sample mean to be roughly normally distributed — which the central limit theorem delivers for most data by about n = 30, but not for strongly skewed data or data with outliers at small n. If a single extreme value is driving your result, a rank-based alternative such as the Mann-Whitney family is more robust.
Critical values of t
| df | One-tailed α = 0.05 | Two-tailed α = 0.05 | Two-tailed α = 0.01 |
|---|---|---|---|
| 5 | 2.015 | 2.571 | 4.032 |
| 9 | 1.833 | 2.262 | 3.250 |
| 10 | 1.812 | 2.228 | 3.169 |
| 14 | 1.761 | 2.145 | 2.977 |
| 19 | 1.729 | 2.093 | 2.861 |
| 24 | 1.711 | 2.064 | 2.797 |
| 29 | 1.699 | 2.045 | 2.756 |
| 49 | 1.677 | 2.010 | 2.680 |
| 99 | 1.660 | 1.984 | 2.626 |
| ∞ | 1.645 | 1.960 | 2.576 |
Standard published Student t table. The bottom row is the normal distribution, which t approaches as df grows.
Ways a t-test goes wrong
- Choosing the tail after seeing the data. A one-tailed test picked because the difference went that way is a two-tailed test with the alpha secretly doubled.
- Reading a non-significant result as proof of no difference. Absence of evidence is not evidence of absence; the confidence interval tells you how much difference you have failed to rule out.
- Testing many hypotheses and reporting the significant one. Twenty independent tests of true nulls produce, on average, one p-value below 0.05. Adjust for multiplicity or pre-register the primary outcome.
- Treating paired data as one sample of independent values. Before-and-after measurements on the same subjects belong in a paired t-test, which tests the mean of the differences and is far more powerful.
- Entering the standard error where the standard deviation belongs. The field wants s. Supplying an already-divided value inflates t by a factor of √n.
- Ignoring the effect size. With n = 10,000, a difference of no practical consequence will be significant. Report Cohen's d or the raw difference with its interval alongside every p-value.
Which test to use instead
If you are comparing two independent groups rather than one group against a fixed number, use the two-sample t-test calculator. If the same subjects were measured twice, use the paired t-test calculator — which is really this one-sample test applied to the within-subject differences. For three or more groups, run a one-way ANOVA rather than a series of pairwise t-tests, which would inflate the false-positive rate.
If your outcome is a proportion rather than a mean, the t-test is the wrong tool; use a proportion test. If the data are ordinal, badly skewed, or dominated by outliers, a rank-based test makes fewer assumptions. And if you have a test statistic from elsewhere and simply want the tail area, the p-value calculator converts z, t, chi-square and F statistics directly.
Before running a study, decide how large a difference matters and check whether your planned sample can detect it. The statistical power calculator and the Cohen's d calculator turn that into a concrete number, and doing it in advance is far more useful than explaining a null result afterwards.
Key terms
- Null hypothesis (H₀)
- The claim that the population mean equals μ₀. The test never proves it; it either rejects it or fails to.
- p-value
- The probability of a result at least as extreme as the one observed, computed assuming H₀ is true.
- Alpha (α)
- The false-positive rate you accept in advance. Reject H₀ when p < α.
- Cohen's d
- The difference expressed in standard deviations rather than standard errors. Unlike t, it does not grow with sample size.