What least squares actually fits
Given a cloud of paired points, infinitely many straight lines could be drawn through it. Least squares picks one by a specific rule: minimise the sum of the squared vertical distances from each point to the line. Those distances are the residuals, and squaring them means large misses count disproportionately while the sign of the miss does not matter.
The rule is asymmetric on purpose. Distances are measured vertically, in the y direction only, because the model treats x as known and y as the quantity carrying the error. Swap the roles of x and y and you get a different line — regressing height on weight is not the inverse of regressing weight on height. If both variables carry measurement error and neither is naturally the predictor, ordinary least squares is the wrong tool, and correlation is the symmetric summary you probably want.
Two facts fall out of the minimisation and are worth carrying around. The fitted line always passes through the point of means (x̄, ȳ). And the residuals always sum to zero whenever the model includes an intercept. Neither is evidence that the fit is any good; both are consequences of the arithmetic.
Where the slope formula comes from
Write the residual for point i as yᵢ − (a + bxᵢ), square it, sum over all points, and minimise with respect to a and b. Setting both partial derivatives to zero gives the normal equations, whose solution is b = Sₓᵧ/Sₓₓ and a = ȳ − b·x̄.
The numerator Sₓᵧ = Σ(x−x̄)(y−ȳ) is the co-movement: it is positive when x and y tend to sit on the same side of their means, negative when they sit on opposite sides, and near zero when there is no consistent pattern. The denominator Sₓₓ = Σ(x−x̄)² measures how spread out the x values are. Dividing one by the other converts co-movement into a rate of change in the units of y per unit of x.
The denominator also explains a practical fact: a design with widely spread x values estimates the slope more precisely than one where every x sits close together. That shows up directly in the standard error of the slope, sₑ/√Sₓₓ, which shrinks as the x values spread out. When you can choose where to sample, spread the x values.
R² compares the leftover variation to the original variation. SST = Σ(y−ȳ)² is how much y varies about its own mean; SSE = Σ(y−ŷ)² is how much it still varies about the line. R² = 1 − SSE/SST is the fraction removed. In simple regression with one predictor it is exactly the square of the correlation coefficient, which is why an r of 0.7745967 and an R² of 0.6 describe the same fit.
The residual standard error, √(SSE/(n−2)), is the typical vertical miss, in the units of y. Two degrees of freedom are subtracted because you estimated two parameters. It is the number to quote when someone asks how accurate a prediction from the line will be — R² is unitless and does not answer that question.
Worked example: five points, fitted by hand
Take the calculator's default data: x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5.
- Means. x̄ = 15/5 = 3; ȳ = 20/5 = 4.
- Deviations. x − x̄ = −2, −1, 0, 1, 2. y − ȳ = −2, 0, 1, 0, 1.
- Sₓᵧ. (−2)(−2) + (−1)(0) + (0)(1) + (1)(0) + (2)(1) = 4 + 0 + 0 + 0 + 2 = 6.
- Sₓₓ. 4 + 1 + 0 + 1 + 4 = 10.
- Slope. b = 6/10 = 0.6.
- Intercept. a = 4 − 0.6 × 3 = 4 − 1.8 = 2.2. The equation is ŷ = 2.2 + 0.6x.
- Fitted values. 2.8, 3.4, 4.0, 4.6, 5.2. Residuals: −0.8, +0.6, +1.0, −0.6, −0.2 — which sum to zero, as they must.
- SSE. 0.64 + 0.36 + 1.00 + 0.36 + 0.04 = 2.4.
- SST. Sᵧᵧ = 4 + 0 + 1 + 0 + 1 = 6.
- R². 1 − 2.4/6 = 0.6. Correlation r = 6/√(10×6) = 6/7.745967 = 0.7746, and 0.7746² = 0.6 as expected.
- Residual standard error. √(2.4/3) = √0.8 = 0.8944.
- Prediction at x = 6. ŷ = 2.2 + 0.6 × 6 = 5.8 — but x = 6 lies beyond the data, so this is extrapolation.
The residual standard error puts the fit in context: the line explains 60% of the variation in y, and a typical point still sits about 0.89 units away from it. With five points and a residual error that size, the slope's own standard error is 0.8944/√10 = 0.283, so a 95% interval for the slope runs 0.6 ± 3.182 × 0.283 = −0.30 to 1.50 using t(0.025, 3) = 3.182. The slope is not distinguishable from zero on this much data.
Reading slope, intercept and R²
The slope carries the meaning. It is the estimated change in y for a one-unit increase in x, in real units — dollars per hour, millimetres per degree, conversions per thousand impressions. Quote it with its standard error or confidence interval; a slope of 0.6 ± 0.28 says something very different from 0.6 ± 0.02.
The intercept is often meaningless on its own. It is the fitted y at x = 0, which is an extrapolation whenever your data do not include x near zero. A regression of house price on floor area will produce an intercept that describes a house of zero area, and no one should read it as a price.
R² is a proportion, not a verdict. It is the share of variation in y that the line accounts for: 0.25 means the line removes a quarter of the variance and 0.75 remains unexplained. What counts as high depends entirely on the field — an R² of 0.3 is strong for individual human behaviour and would indicate a broken instrument in a calibration laboratory. R² also cannot fall when you add predictors, which is why multiple regression reports the adjusted R² instead.
The residuals decide whether the model is right. Look at the residual column in the table. If it drifts from negative to positive to negative as x increases, the true relationship is curved and a straight line is misspecified regardless of R². If the residuals fan out as x grows, the error variance is not constant and the standard errors are unreliable. Anscombe's quartet — four datasets with identical slope, intercept and R² but completely different shapes — exists precisely to make this point.
The worked example's residual table
| x | y | ŷ = 2.2 + 0.6x | Residual | Residual² | (y − ȳ)² |
|---|---|---|---|---|---|
| 1 | 2 | 2.8 | −0.8 | 0.64 | 4 |
| 2 | 4 | 3.4 | 0.6 | 0.36 | 0 |
| 3 | 5 | 4.0 | 1.0 | 1.00 | 1 |
| 4 | 4 | 4.6 | −0.6 | 0.36 | 0 |
| 5 | 5 | 5.2 | −0.2 | 0.04 | 1 |
| Sum | 20 | 20.0 | 0.0 | 2.40 | 6 |
The residual column sums to zero and the fitted values sum to the same total as the observed values — both are guaranteed by least squares with an intercept. SSE = 2.40 and SST = 6 give R² = 0.6.
Assumptions and traps
- Linearity. The method fits a straight line whether or not the relationship is straight. Check the residual column before believing the slope.
- Independent errors. Time-series data usually violates this: consecutive residuals are correlated, and the standard errors come out far too small.
- Constant error variance. If the spread of y grows with x, the fit is still unbiased but the standard errors and any interval built from them are not trustworthy.
- Extrapolation. A prediction outside the observed range of x assumes the line continues where you have no data. The calculator warns when your prediction point leaves the range.
- Influential points. A single observation with an extreme x can pivot the whole line. Re-run without it and see whether the slope survives.
- Correlation is not causation. A significant slope on observational data establishes association. Only the design — randomisation, controls, timing — supports a causal reading.
Related tools
If you want the strength and direction of association without a directional model, use the Pearson correlation calculator, which also tests whether r differs from zero and gives a confidence interval for it. For simple regression the two are tightly linked: R² = r², and the t test for the slope is exactly the t test for the correlation.
For monotonic but non-linear relationships, or data with outliers, Spearman's rank correlation works on ranks instead of raw values. For the raw co-movement in original units rather than a standardised one, see the covariance calculator. When you move to several predictors, the right summary of fit becomes adjusted R², which penalises added terms.
Least squares dates to Legendre's 1805 publication and Gauss's independent development around the same period, both driven by the problem of fitting orbits to noisy astronomical observations. The method survived two centuries of alternatives because, under the standard assumptions, it produces the minimum-variance unbiased estimates of the slope and intercept — the Gauss-Markov theorem.