Regression, financial models and residuals
Introduzido| English | Significado |
|---|---|
| ranks | The positions of values when they are put in order: 1st, 2nd, 3rd and so on. |
| Spearman's rank correlation coefficient/ˈspɪəmənz ræŋk ˌkɒrɪˈleɪʃn ˌkəʊɪˈfɪʃənt/ | A measure of monotonic association obtained by correlating the ranks of paired observations. |
| bivariate normal | Describes two variables whose joint distribution is normal, the condition for testing the product moment correlation coefficient. |
| monotonic | Describes a relationship in which one variable always increases, or always decreases, as the other increases. |
| residual/rɪˈsɪdʒuːəl/ | resíduo |
Spearman's ranks, 1904
- In 1904, the English psychologist Charles Spearman published "The proof and measurement of association between two things". He wanted to compare children's rankings in different school subjects, where the data were orders, not measurements, so he built a correlation coefficient from ranks 秩.
- S3 uses Spearman's rank correlation coefficient 斯皮尔曼等级相关系数 and the product moment correlation coefficient in hypothesis tests.
Spearman's coefficient
- Rank each variable, find each difference $d$ between the two ranks, and use $r_s = 1 - \dfrac{6\sum d^2}{n(n^2 - 1)}$.
- $r_s = 1$ means the orders agree exactly and $r_s = -1$ that they are reversed. Tied values each get the mean of the ranks they share.
For 5 pairs of ranks, $\sum d^2 = 2$. Find $r_s$.
Dificuldade 2/5 · Moedas 140 + pequeno bônus na primeira conclusão hoje
$1 - \dfrac{12}{5 \times 24} = 0.9$.
Two values tie for 3rd and 4th place. What rank does each get?
Dificuldade 1/5 · Moedas 100 + pequeno bônus na primeira conclusão hoje
The mean of the ranks they share.
Testing for correlation
- $\mathrm{H}_0$: $\rho = 0$ (no correlation in the population); $\mathrm{H}_1$: $\rho > 0$, $\rho < 0$ or $\rho \ne 0$. Compare $r$ or $r_s$ with the critical value for $n$ and the significance level.
- For the product moment coefficient with $n = 10$, the 5% one-tailed critical value is 0.5494: $r = 0.62$ gives evidence of positive correlation.
A researcher asks whether there is any correlation between two variables. What is $\mathrm{H}_1$?
Dificuldade 1/5 · Moedas 100 + pequeno bônus na primeira conclusão hoje
"Any" correlation is two-tailed; hypotheses are about the population $\rho$, not the sample $r$.
Worked example: two judges
Two judges rank six singers. Judge A: 1, 2, 3, 4, 5, 6. Judge B: 2, 1, 3, 4, 6, 5. Test at the 5% level whether the judges agree (critical value for $n = 6$, one-tailed: 0.8286).
- Differences: $d = -1, 1, 0, 0, -1, 1$, so $\sum d^2 = 4$.
- Coefficient: $r_s = 1 - \dfrac{6 \times 4}{6 \times 35} = 1 - \dfrac{24}{210} = 0.886$.
- Decision: $\mathrm{H}_0$: $\rho = 0$, $\mathrm{H}_1$: $\rho > 0$. Since $0.886 > 0.8286$, reject $\mathrm{H}_0$: there is evidence that the judges' rankings agree.
Two judges rank 6 items; the rankings are exactly reversed. Find $r_s$.
Dificuldade 2/5 · Moedas 140 + pequeno bônus na primeira conclusão hoje
$\sum d^2 = 25 + 9 + 1 + 1 + 9 + 25 = 70$; $1 - \tfrac{420}{210} = -1$.
Which coefficient?
- Use the product moment coefficient for a linear relationship when the data are measurements from a bivariate normal 二元正态 population.
- Use Spearman when the data are ranks, or when the relationship is monotonic 单调的 but not linear: $y = x^3$ on $1 \le x \le 5$ has $r_s = 1$ but $r < 1$.
If $y$ always increases with $x$ but not along a straight line, Spearman's coefficient can be 1 while the product moment coefficient is less than 1.
Dificuldade 1/5 · Moedas 100 + pequeno bônus na primeira conclusão hoje
Spearman measures monotonic order; the product moment coefficient measures straight-line fit.
Correlation, regression and residuals
- A significant correlation only shows association. A regression line is then used for prediction, and residuals 残差 (observed − predicted) show how well it fits.
- With $t = 8 + 2d$, a 5 km delivery predicted at 18 minutes that took 21 has residual 3.
Regression, financial models and residuals
A linear model y=a+bx has intercept a and slope b with contextual units
Use the positive residual to explain how the fitted model misses this observation.
With $t = 8 + 2d$, a 4 km delivery took 14 minutes. Find the residual.
Dificuldade 2/5 · Moedas 140 + pequeno bônus na primeira conclusão hoje
Predicted 16; $14 - 16 = -2$.
Using a two-tailed critical value for a one-tailed test. "Is there positive correlation?" is one-tailed; "is there any correlation?" is two-tailed, and at the same significance level its critical value is larger. Choose $\mathrm{H}_1$ from the question before you look up the table.
Spearman's $r_s = 1 - \tfrac{6\sum d^2}{n(n^2 - 1)}$ measures agreement between rankings; tied values share mean ranks. Test $\mathrm{H}_0$: $\rho = 0$ by comparing $r$ or $r_s$ with the critical value for $n$, the level and the number of tails. Use the product moment coefficient for linear, bivariate normal data, and Spearman for ranks or monotonic links.
For 8 students, ranks in maths and physics give $\sum d^2 = 18$. Test at the 5% level whether there is positive association (critical value 0.6429).
Coefficient, compare, conclude.