Regression and classification

Vocabulaire

English	Sens
linear regression/ˈlɪnɪə rɪˈɡreʃn/	A method of modelling a response using a linear combination of explanatory variables and fitted coefficients.
logistic regression	A model mapping a linear score through a logistic function to estimate a binary-class probability.
decision boundary	The boundary between input regions assigned different predicted classes.

English

Lines and boundaries

Linear regression fits $\hat{y} = wx + b$ by least squares; logistic regression fits a probability boundary with the sigmoid; the decision boundary is where the model changes its answer.

{exercise:uni-cs1304-models}

Pratique

The decision boundary is where —

Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui

English

Course points $(1,2), (2,3), (3,5), (4,4), (5,6)$: $\bar{x} = 3$, $\bar{y} = 4$, $w = 9/10 = 0.9$, $b = 1.3$ — residuals $-0.2, -0.1, 1.0, -0.9, 0.2$, SSE $1.9$ (every number suite-verified).
Logistic boundary with $w_1 = 2, w_2 = -1, b = -1$: the line $x_2 = 2x_1 - 1$; a parcel at $(2, 2)$ scores $+1$ (late), at $(2, 5)$ scores $-2$ (on time); the margin is the score's magnitude.

{exercise:uni-cs1304-leastsquares}

Pratique

The course five points fit the line —

Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui

English

A 4th-degree polynomial hits SSE $0$ on the five training points but SSE $\approx 42.7$ on the five held-out points — against the line's $0.55$. Training error is an advertisement; held-out error is the product.
Model choice by held-out error; and fitted weights are not causes (unit 8's confounding lives here too).

{exercise:uni-cs1304-overfit}

Pratique

The 4th-degree polynomial on held-out data scores SSE —

Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui