What sensitivity and specificity actually describe
Sensitivity and specificity describe how a test behaves within each disease group. Sensitivity asks: of the people who genuinely have the condition, what share does the test catch? Specificity asks: of the people who genuinely do not, what share does the test correctly clear? Each denominator runs down one column of the 2×2 table, and that is the property that makes the pair useful.
Because the denominators are within-group, neither figure depends on how many of each group you recruited. A validation study that enrols 200 confirmed cases and 800 healthy volunteers gives the same sensitivity as one that enrols 50 and 950. That is not true of accuracy, and it is emphatically not true of predictive values, which move enormously with the prevalence in the tested group.
The two measures trade against each other. Every test with a continuous readout has a cut-off, and moving the cut-off to catch more cases inevitably flags more healthy people as well. Sensitivity and specificity are therefore a pair of coordinates on that trade-off curve, not two independent virtues. Quoting one without the other is meaningless: a test that reports “positive” for everyone has 100% sensitivity.
Clinicians remember the trade-off with two mnemonics. SnNout: a highly Snsitive test, when Negative, rules the condition out. SpPin: a highly Specific test, when Positive, rules it in. Both are approximations that hold when the corresponding rate is near 100%, and both are better expressed as likelihood ratios.
The formulas, the Youden index, and why the interval is Wilson's
The two point estimates are simple proportions. The Youden index J = Se + Sp − 1 collapses them into one number that is 0 when the test is no better than a coin weighted to the prevalence, and 1 when it is perfect. J is the vertical distance from a point on the ROC curve to the diagonal, which is why it is the standard criterion for choosing an optimal cut-off when false positives and false negatives are considered equally costly. When they are not equally costly, J is the wrong criterion and you should weight the two errors explicitly.
The interval is where the choices matter. The textbook interval for a proportion, p̂ ± z·√(p̂(1−p̂)/n), fails precisely where diagnostic tests live. At p̂ = 1.00 it has zero width, which asserts perfect certainty from any sample size at all; near 0 or 1 it routinely extends outside the range 0 to 1; and its actual coverage at nominal 95% can drop well below 95% even at moderate n.
Wilson's score interval, published in 1927, fixes all three problems. Instead of centring on p̂, it solves for the values of p at which the score test would just reach significance. The result is an interval shrunk towards 0.5, always inside 0 and 1, non-degenerate at the boundaries, and with coverage close to nominal even for small n. Newcombe's 1998 comparison of seven methods is the standard reference for why it is preferred, and it is the interval this calculator reports.
Worked example: 100 confirmed cases and 900 confirmed controls
A new rapid assay is validated against a reference standard. Among 100 people confirmed to have the condition, the assay flags 90. Among 900 confirmed free of it, the assay flags 20.
- Fill the table. TP = 90, FN = 10, FP = 20, TN = 880. N = 1,000.
- Sensitivity. 90 ÷ (90 + 10) = 90 ÷ 100 = 90.0%.
- Specificity. 880 ÷ (880 + 20) = 880 ÷ 900 = 97.78%.
- Accuracy. (90 + 880) ÷ 1,000 = 97.0%.
- Youden index. 0.900 + 0.9778 − 1 = 0.878.
- False negative rate. 10 ÷ 100 = 10.0%. False positive rate. 20 ÷ 900 = 2.22%.
Now the Wilson interval for sensitivity, with p̂ = 0.90, n = 100 and z = 1.95996 (so z² = 3.84146):
- Denominator: 1 + 3.84146 ÷ 100 = 1.038415.
- Centre: (0.90 + 3.84146 ÷ 200) ÷ 1.038415 = 0.919207 ÷ 1.038415 = 0.885205.
- Inside the root: (0.90 × 0.10) ÷ 100 + 3.84146 ÷ 40,000 = 0.000900 + 0.000096 = 0.000996. Its square root is 0.031560.
- Half-width: (1.95996 × 0.031560) ÷ 1.038415 = 0.061856 ÷ 1.038415 = 0.059568.
- Limits: 0.885205 ∓ 0.059568 = 0.8256 to 0.9448, i.e. 82.6% to 94.5%.
Notice that the interval is not centred on 0.90 — the centre sits at 0.885, pulled towards 0.5. That shrinkage is the whole point of the method, and it is why the Wilson interval keeps its coverage where the normal approximation loses it.
How to judge the numbers you get
Judge each figure against the job the test has to do. A triage or rule-out test needs high sensitivity, because a missed case leaves the patient unmanaged; specificity can be sacrificed because a confirmatory test follows. A confirmatory test needs high specificity, because a false positive now leads directly to treatment. Screening programmes usually chain the two, which is why the sensitivity of the first stage and the specificity of the second stage are the numbers that matter most.
Read the interval, not just the estimate. The single most under-appreciated fact about these studies is that precision on sensitivity is bought with cases, not with controls. A study with 20 confirmed cases and 5,000 controls will report a sensitivity interval roughly 27 percentage points wide however impressive the control arm looks. The reference table below quantifies that, and it is the number to check first when you read someone else's validation study.
Overall accuracy deserves suspicion. It is a prevalence-weighted average of sensitivity and specificity, so in a rare condition it is dominated by specificity and can look excellent for a test that catches almost nothing. In the worked example above accuracy is 97% — but a test that simply reported “negative” for all 1,000 subjects would have scored 90%. Report sensitivity and specificity separately; treat accuracy as a footnote.
How wide is the interval? Wilson 95% CI for an observed 90% sensitivity
| Confirmed cases (n) | Observed Se | Lower limit | Upper limit | Width |
|---|---|---|---|---|
| 10 | 90.0% | 59.6% | 98.2% | 38.6 pp |
| 20 | 90.0% | 69.9% | 97.2% | 27.3 pp |
| 50 | 90.0% | 78.6% | 95.7% | 17.0 pp |
| 100 | 90.0% | 82.6% | 94.5% | 11.9 pp |
| 200 | 90.0% | 85.1% | 93.4% | 8.4 pp |
| 500 | 90.0% | 87.1% | 92.3% | 5.3 pp |
Widths are in percentage points. Halving the width takes roughly four times the number of cases, which is the usual square-root behaviour of a standard error.
Pitfalls that inflate or deflate these numbers
- Spectrum bias. Validating on hospitalised, advanced cases and on healthy blood donors makes both figures look far better than they will be in the population where the test is actually used.
- An imperfect reference standard. If the gold standard itself misclassifies, your table is contaminated. A new test that disagrees with a flawed reference is penalised for being right.
- Verification bias. Sending only test-positive subjects for confirmation inflates sensitivity and deflates specificity, because the false negatives are never verified and never enter the table.
- Discarding indeterminate results. Real assays produce equivocal readings. A 2×2 table has no cell for them, and dropping them silently raises every performance measure.
- Choosing the cut-off on the same data you evaluate. Picking the threshold that maximises the Youden index and then reporting that maximum overstates performance. Split the data or cross-validate.
- Quoting accuracy for a rare condition. It is a prevalence-weighted average and can be dominated by whichever group is larger.
- Counting a paired or clustered design as independent. Two lesions in one patient are not two independent observations, and the Wilson interval will be too narrow if you treat them as such.
Where these numbers go next
Sensitivity and specificity are intermediate quantities. Almost nothing you want to do clinically uses them directly.
To answer “this patient tested positive — what now?” you need a prevalence and Bayes' theorem, which produces the positive predictive value. The PPV and NPV calculator does that conversion and shows how violently the answer moves with prevalence. To express the same information in a prevalence-free form you can apply to any patient, convert to likelihood ratios with the diagnostic likelihood ratio calculator: LR+ = Se ÷ (1 − Sp) and LR− = (1 − Se) ÷ Sp, and their ratio is the diagnostic odds ratio computed by the odds ratio calculator.
When you need an interval for some other proportion in your study — a subgroup sensitivity, a completion rate, a positivity rate — the confidence interval for a proportion calculator applies the same Wilson method. And if the clinical question is what a treatment decision buys after the diagnosis is made, the number needed to treat calculator converts trial event rates into patients treated per event avoided.
Finally, if you are writing this up, work through the STARD 2015 checklist. It asks for the flow of participants, how the reference standard was applied, how indeterminate results were handled, and the full 2×2 table with estimates of precision — which is to say, exactly the numbers this page produces plus the study design that justifies them.
Key terms
- Reference standard
- The best available method for establishing whether the condition is truly present. Everything in the table is defined relative to it.
- Youden index (J)
- Se + Sp − 1. The vertical distance from a point on the ROC curve to the chance diagonal; used to pick an optimal cut-off.
- Wilson score interval
- A confidence interval for a proportion obtained by inverting the score test. Stays within 0 and 1 and behaves well near the boundaries.
- Spectrum bias
- Bias caused by measuring accuracy on a case mix more advanced, or a control mix healthier, than the intended use population.
- Verification bias
- Bias arising when the reference standard is applied selectively, typically only to subjects who tested positive.