Regression, financial models and residuals
引入| English | 释义 | 拼音 |
|---|---|---|
| ranks | 秩 | zhì |
| Spearman's rank correlation coefficient/ˈspɪəmənz ræŋk ˌkɒrɪˈleɪʃn ˌkəʊɪˈfɪʃənt/ | 斯皮尔曼等级相关系数 | sī pí ěr màn děng jí xiāng guān xì shù |
| bivariate normal | 二元正态 | èr yuán zhèng tài |
| monotonic | 单调的 | dān diào de |
| residual/rɪˈsɪdʒuːəl/ | 残差 | cán chà |
Spearman's ranks, 1904
- In 1904, the English psychologist Charles Spearman published "The proof and measurement of association between two things". He wanted to compare children's rankings in different school subjects, where the data were orders, not measurements, so he built a correlation coefficient from ranks 秩.
- S3 uses Spearman's rank correlation coefficient 斯皮尔曼等级相关系数 and the product moment correlation coefficient in hypothesis tests.
Spearman's coefficient
- Rank each variable, find each difference $d$ between the two ranks, and use $r_s = 1 - \dfrac{6\sum d^2}{n(n^2 - 1)}$.
- $r_s = 1$ means the orders agree exactly and $r_s = -1$ that they are reversed. Tied values each get the mean of the ranks they share.
For 5 pairs of ranks, $\sum d^2 = 2$. Find $r_s$.
难度 2/5 · 今日首次完成可获 140 金币及少量额外奖励
$1 - \dfrac{12}{5 \times 24} = 0.9$.
Two values tie for 3rd and 4th place. What rank does each get?
难度 1/5 · 今日首次完成可获 100 金币及少量额外奖励
The mean of the ranks they share.
Testing for correlation
- $\mathrm{H}_0$: $\rho = 0$ (no correlation in the population); $\mathrm{H}_1$: $\rho > 0$, $\rho < 0$ or $\rho \ne 0$. Compare $r$ or $r_s$ with the critical value for $n$ and the significance level.
- For the product moment coefficient with $n = 10$, the 5% one-tailed critical value is 0.5494: $r = 0.62$ gives evidence of positive correlation.
A researcher asks whether there is any correlation between two variables. What is $\mathrm{H}_1$?
难度 1/5 · 今日首次完成可获 100 金币及少量额外奖励
"Any" correlation is two-tailed; hypotheses are about the population $\rho$, not the sample $r$.
Worked example: two judges
Two judges rank six singers. Judge A: 1, 2, 3, 4, 5, 6. Judge B: 2, 1, 3, 4, 6, 5. Test at the 5% level whether the judges agree (critical value for $n = 6$, one-tailed: 0.8286).
- Differences: $d = -1, 1, 0, 0, -1, 1$, so $\sum d^2 = 4$.
- Coefficient: $r_s = 1 - \dfrac{6 \times 4}{6 \times 35} = 1 - \dfrac{24}{210} = 0.886$.
- Decision: $\mathrm{H}_0$: $\rho = 0$, $\mathrm{H}_1$: $\rho > 0$. Since $0.886 > 0.8286$, reject $\mathrm{H}_0$: there is evidence that the judges' rankings agree.
Two judges rank 6 items; the rankings are exactly reversed. Find $r_s$.
难度 2/5 · 今日首次完成可获 140 金币及少量额外奖励
$\sum d^2 = 25 + 9 + 1 + 1 + 9 + 25 = 70$; $1 - \tfrac{420}{210} = -1$.
Which coefficient?
- Use the product moment coefficient for a linear relationship when the data are measurements from a bivariate normal 二元正态 population.
- Use Spearman when the data are ranks, or when the relationship is monotonic 单调的 but not linear: $y = x^3$ on $1 \le x \le 5$ has $r_s = 1$ but $r < 1$.
If $y$ always increases with $x$ but not along a straight line, Spearman's coefficient can be 1 while the product moment coefficient is less than 1.
难度 1/5 · 今日首次完成可获 100 金币及少量额外奖励
Spearman measures monotonic order; the product moment coefficient measures straight-line fit.
Correlation, regression and residuals
- A significant correlation only shows association. A regression line is then used for prediction, and residuals 残差 (observed − predicted) show how well it fits.
- With $t = 8 + 2d$, a 5 km delivery predicted at 18 minutes that took 21 has residual 3.
Regression, financial models and residuals
A linear model y=a+bx has intercept a and slope b with contextual units
Use the positive residual to explain how the fitted model misses this observation.
With $t = 8 + 2d$, a 4 km delivery took 14 minutes. Find the residual.
难度 2/5 · 今日首次完成可获 140 金币及少量额外奖励
Predicted 16; $14 - 16 = -2$.
Using a two-tailed critical value for a one-tailed test. "Is there positive correlation?" is one-tailed; "is there any correlation?" is two-tailed, and at the same significance level its critical value is larger. Choose $\mathrm{H}_1$ from the question before you look up the table.
Spearman's $r_s = 1 - \tfrac{6\sum d^2}{n(n^2 - 1)}$ measures agreement between rankings; tied values share mean ranks. Test $\mathrm{H}_0$: $\rho = 0$ by comparing $r$ or $r_s$ with the critical value for $n$, the level and the number of tails. Use the product moment coefficient for linear, bivariate normal data, and Spearman for ranks or monotonic links.
For 8 students, ranks in maths and physics give $\sum d^2 = 18$. Test at the 5% level whether there is positive association (critical value 0.6429).
Coefficient, compare, conclude.