Regression and AI in practice
| English | Chinese | Pinyin |
|---|---|---|
| regression | 回归 | huí guī |
| classification | 分类 | fēn lèi |
| linear regression | 线性回归 | xiàn xìng huí guī |
| OCR | 光学字符识别 | guāng xué zì fú shí bié |
Predicting a number, not a name
- Ask a model "is this email spam?" and it must choose from a fixed set of answers. Ask it "what will this house sell for?" and there is no set to choose from: the answer is a number, and being out by a thousand pounds is nearly right.
- Those are two different tasks, and mixing them up is the commonest mistake in this topic. Both are supervised learning; only the shape of the answer differs.
- This lesson is regression 回归 against classification 分类, how linear regression 线性回归 chooses its line, and how real products chain trained models into a pipeline.
Regression and classification
- Classification predicts a category from a fixed set: cat or dog, spam or not spam, which of ten digits.
- Regression predicts a number on a continuous scale: a house price, tomorrow's temperature, how long a journey will take.
- Both are supervised: both learn from labelled examples. The choice between them is decided by one question: is the answer a label or a number?
- The distinction changes how you measure success. A classifier is right or wrong; a regressor is a certain distance out, which is why their loss functions differ.
The difference between regression and classification is that regression predicts:
Regression outputs a continuous number (price, temperature); classification outputs a category.
Regression predicts a continuous number (a price, a temperature), while classification predicts a category (cat vs dog).
Number out = regression; category out = classification — the two main kinds of supervised prediction.
Worked example: which task is which
- Predicting which of five customer types a shopper belongs to. Classification: the answer is one of a fixed set of categories.
- Predicting how much that shopper will spend next month. Regression: the answer is a number, and being close counts as nearly right.
- Predicting whether a machine will fail in the next week. Classification: yes or no. Predicting how many days until it fails. Regression.
- The same underlying data can feed either. Read the question being asked, not the subject matter.
Match each prediction task to its kind.
The same data can feed either. Read the question: a label from a fixed set, or a number on a scale.
Linear regression
- Linear regression fits a straight line, or a flat surface when there are several inputs:
- The coefficients are chosen to minimise the sum of the squared errors between the line's predictions and the actual training values.
- Squared, for two reasons the exam accepts: errors above and below the line then cannot cancel out, and a large error is penalised much more than several small ones.
The line that leaves the least total squared distance
Fitting a regression line
Drag the controls. Linear regression draws the straight line that makes the squared distances to the data points as small as possible — then it predicts a number for any new input.
Linear regression chooses its coefficients to:
It fits the line/hyperplane that makes the squared differences from the data points as small as possible.
Why does linear regression minimise the sum of the SQUARED errors rather than the errors themselves?
Both reasons are accepted. Without squaring, a line could have a huge error above and below and still score zero.
When a straight line is the right model
- Use linear regression when the relationship looks roughly linear and you want a model a human can read: each coefficient says how much that input matters, and in which direction.
- When the data curves, the same idea still applies with a different model: polynomial regression, a decision tree, or a neural network. In every case the recipe is the same, choose a model, define a loss, minimise it.
- That recipe is what connects this lesson to the last one: gradient descent is simply how the minimising is done when there is no formula for the answer.
AI in a real product
- Real systems rarely use one model. They chain several trained models into a pipeline, each doing one step.
- Face identification at a door: a camera captures a face, image recognition converts it into a numerical representation, and that is matched against the representations of registered people.
- Reading a sign aloud for a traveller: image recognition locates the text, OCR 光学字符识别 turns the pixels into characters, machine translation converts the words, and text-to-speech produces the audio.
- The exam gives a scenario and asks which models are used, in order. Name the stage and what it produces for the next one.
Match each AI idea to what it means.
Training fits the model once; inference is the quick prediction each time a user interacts.
A "read a sign aloud" feature typically chains which models?
Several trained models form a pipeline: find the text, extract characters, translate, then speak.
Put the stages of an app that reads a foreign sign aloud in order.
Several trained models chained together, each handing its output to the next. Name the stage and what it produces.
Worked example: training against inference
- A phone app recognises a plant from a photograph instantly, yet training its model took weeks on many machines. Explain why.
- Training repeatedly runs forward and backward passes over a very large labelled dataset, adjusting millions of weights over many epochs. That is what took weeks.
- Inference, using the trained model, is a single forward pass through a network whose weights are now fixed: multiply, add, activate, layer by layer, once.
- The intelligence lives in the learned weights, which is why the finished model is small enough and fast enough to ship on a phone.
Using a trained model to make a prediction requires only a single ____ pass.
Training repeats forward and backward passes over a huge dataset; inference runs the fixed weights once, which is why it is instant.
Marks that slip away
- Classification gives a category, regression gives a number. Both are supervised; that is not the difference.
- Linear regression minimises the sum of squared errors, not "the total distance". Say squared, and say why.
- A pipeline answer must give the stages in order and say what each hands to the next.
- Training is repeated forward and backward passes; inference is one forward pass. Do not describe training when the question asks about use.
You've got it
- classification predicts a category, regression predicts a number; both are supervised, and the question's answer shape decides which
- linear regression fits $y = m_1x_1 + \ldots + c$ by minimising the sum of squared errors, so errors cannot cancel and large ones are penalised heavily
- curved data needs another model, but the recipe is unchanged: choose a model, define a loss, minimise it
- real products chain models: image recognition, OCR, translation and text-to-speech; training is many forward and backward passes, inference is one forward pass