Regression, financial models and residuals
Introduced| English | Meaning |
|---|---|
| regression line/rɪˈɡreʃn laɪn/ | The straight line y = a + bx that best fits a scatter of points, used to predict y from x. |
| product moment correlation coefficient | The number r = Sxy/√(SxxSyy) that measures the strength of a linear relationship, from −1 to 1. |
| residual/rɪˈsɪdʒuːəl/ | An observed value minus the value predicted by a model. |
| coding/ˈkəʊdɪŋ/ | Changing data by a rule such as x = (u − a)/b to make calculation easier. |
Four data sets, one line, 1973
- In 1973, the statistician Francis Anscombe published four small data sets. All four have the same mean of $x$ (9), the same mean of $y$ (7.50), the same correlation (0.816) and the same regression line, $y = 3.00 + 0.500x$. But plotted, one is a curve, one has an outlier and one is a vertical line with a single far point. Anscombe's point: always look at the scatter diagram first.
- In S1 you calculate the product moment correlation coefficient 积差相关系数 and the regression line 回归直线 from summary statistics, then interpret them.
Correlation from summary statistics
- With $S_{xx} = \sum x^2 - \dfrac{(\sum x)^2}{n}$, and $S_{yy}$, $S_{xy} = \sum xy - \dfrac{\sum x \sum y}{n}$ in the same way, the coefficient is $r = \dfrac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}$, between $-1$ and $1$.
- Correlation is not causation: ice-cream sales and drownings both rise in summer.
Which value of $r$ shows the strongest linear relationship?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Closest to $\pm 1$.
Countries with more TVs per person have higher life expectancy. What can you conclude?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Correlation is not causation.
The regression line
- The line of $y$ on $x$ is $y = a + bx$ with $b = \dfrac{S_{xy}}{S_{xx}}$ and $a = \bar{y} - b\bar{x}$. The gradient $b$ is the change in $y$ for each extra unit of $x$.
- The intercept $a$ is the value of $y$ at $x = 0$ — meaningful only if $x = 0$ lies inside the data.
For $y = 12 + 0.6x$ (y = mass in kg, x = age in months), what does 0.6 mean?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Gradient = change in $y$ per unit $x$.
Worked example: delivery times
For 12 deliveries, distance $d$ (km, $2 \le d \le 9$) and time $t$ (minutes) give $\bar{d} = 5.5$, $\bar{t} = 19$, $S_{dd} = 50$, $S_{tt} = 225$ and $S_{dt} = 100$. Find $r$ and the regression line of $t$ on $d$, and assess two predictions.
- Correlation: $r = \dfrac{100}{\sqrt{50 \times 225}} = 0.943$ — strong positive linear correlation.
- Line: $b = \dfrac{100}{50} = 2$ and $a = 19 - 2(5.5) = 8$, so $t = 8 + 2d$: each extra kilometre adds about 2 minutes, after 8 minutes of fixed time.
- Predictions: $d = 5$ gives 18 minutes (interpolation, reliable); a delivery that took 21 minutes has residual 残差 $21 - 18 = 3$. $d = 30$ gives 68 — extrapolation, unreliable.
Regression, financial models and residuals
A linear model y=a+bx has intercept a and slope b with contextual units
Use the positive residual to explain how the fitted model misses this observation.
Given $S_{xx} = 16$, $S_{xy} = 48$, $\bar{x} = 3$ and $\bar{y} = 10$, find $a$ in the regression line $y = a + bx$.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
$b = \tfrac{48}{16} = 3$, then $a = 10 - 3 \times 3 = 1$.
A 4 km delivery took 14 minutes. Find its residual using $t = 8 + 2d$.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Observed 14 minus predicted 16.
Coding
- Coding 编码 such as $x = \dfrac{u - 100}{10}$ does not change $r$, because it only shifts and rescales the axes.
- A line found in coded form must be converted back: if $y = 3 + 2x$ with $x = \dfrac{u - 100}{10}$, then $y = 3 + 0.2(u - 100) = 0.2u - 17$.
Coding the data with $x = \dfrac{u - 50}{5}$ changes the value of $r$.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Shifting and rescaling move the points together, so the strength of the linear link is unchanged.
Which variable to predict
- Use the line of $y$ on $x$ to predict $y$; it should not be rearranged to predict $x$.
- The explanatory variable (here, distance) goes on the $x$-axis.
A strong correlation does not mean the line works everywhere. Predicting outside the data range (extrapolation) assumes the pattern continues, and it often does not.
Data were collected for $2 \le d \le 9$. A prediction at $d = 7$ is extrapolation.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
7 is inside the range: interpolation.
Look at the scatter diagram first. Interpret $r$ for strength and direction of a linear link, the gradient as change per unit, and the intercept only if $x = 0$ is in range. A residual is observed minus predicted. Trust interpolation, not extrapolation, and never read causation from correlation.
A student finds $r = 0.78$ between daily mean temperature (°C) and sunshine hours in summer, and the line $s = -3.1 + 0.55T$. Interpret the gradient, then say whether the intercept and a prediction at $T = 35$ are meaningful.
Interpretation, not calculation.