Regression, financial models and residuals
Introduced| English | Meaning |
|---|---|
| regression/rɪˈɡreʃn/ | A statistical method for modelling how a response variable is related to one or more explanatory variables. |
| residual/rɪˈsɪdʒuːəl/ | An observed value minus the value predicted by a model. |
| correlation coefficient/ˌkɒrɪˈleɪʃn ˌkəʊɪˈfɪʃənt/ | r: a measure from −1 to 1 of the direction and strength of LINEAR association. |
Regression towards the mean, 1886
- In 1886 Francis Galton compared the heights of 928 adult children with those of their parents. Tall parents tended to have children who were tall, but less extremely so; short parents, children less short. He called it "regression towards mediocrity" — and the line he fitted gave regression 回归 its name.
- A regression line predicts one variable from another. Its residuals 残差 — observed minus predicted — show how well it fits; the correlation coefficient 相关系数 $r$ measures the strength of a linear relationship.
What did Galton notice about the children of very tall parents?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Regression towards the mean.
Correlation
- Pearson's $r$ lies between $-1$ and $1$: near $\pm1$ strong linear, near 0 weak or none.
- Correlation is not causation: ice-cream sales and drownings both rise in summer.
Which r shows the strongest linear relationship?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Closest to ±1.
A strong correlation proves that one variable causes the other.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
A third factor may drive both.
The regression line
- The least-squares line $y = ax + b$ minimises the sum of squared residuals; the GDC gives $a$, $b$ and $r$.
- Use the $y$-on-$x$ line to predict $y$ from $x$, only within the data range (interpolation).
Regression, financial models and residuals
A linear model y=a+bx has intercept a and slope b with contextual units
Use the positive residual to explain how the fitted model misses this observation.
Predicting within the range of the data is called
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Extrapolation goes outside it.
Residuals
- Residual $= y_{\text{observed}} - y_{\text{predicted}}$. A pattern in the residuals means a straight line is the wrong model.
- Sum of squared residuals compares models: smaller is better for the same data.
A model predicts 18 and the observation is 21. Find the residual.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Observed − predicted.
Worked example: travel time
A delivery firm fits $t = 8 + 2d$ (minutes, km).
- Prediction at $d = 5$: $t = 18$ minutes. Observed: 21. Residual: $+3$ — slower than predicted.
- Interpretation: the gradient 2 means each extra kilometre adds about 2 minutes; the intercept 8 is loading and parking time.
- Extrapolation: predicting for $d = 60$ km (outside the data, 1–8 km) assumes city speeds on a motorway — unreliable.
In t = 8 + 2d, what does the 2 mean?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
The gradient.
Other models
- Exponential, power and sinusoidal regressions on the GDC; choose by the shape of the scatter and the context, then check the residuals.
A high $r$ does not prove the model is right. A curved relationship can give $r = 0.95$ with a clear pattern in its residuals. Look at the residual plot.
If r = 0.95, a straight line must be the right model.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Check the residuals.
Use $r$ for the strength of a linear relationship, never as proof of cause. Fit the least-squares line, interpret its gradient and intercept in context, predict only within the data, and judge fit by residuals: observed minus predicted, with no pattern.
Study hours h and test score s give s = 42 + 5.5h, r = 0.81, for 0 ≤ h ≤ 8. Interpret the gradient, predict for h = 6, and comment on predicting for h = 20.
Interpret, predict, limit.