Machine learning types and training
| English | Chinese | Pinyin |
|---|---|---|
| machine learning | 机器学习 | jī qì xué xí |
| backpropagation | 反向传播 | fǎn xiàng chuán bō |
| supervised learning | 监督学习 | jiān dū xué xí |
| classification | 分类 | fēn lèi |
| unsupervised learning | 无监督学习 | wú jiān dū xué xí |
| clusters | 聚类 | jù lèi |
| reinforcement learning | 强化学习 | qiáng huà xué xí |
| agent | 智能体 | zhì néng tǐ |
| reward | 奖励 | jiǎng lì |
| policy | 策略 | cè lüè |
| gradient descent | 梯度下降 | tī dù xià jiàng |
| loss function | 损失函数 | sǔn shī hán shù |
| epoch | 训练轮次 | xùn liàn lún cì |
The program that taught itself an opening no human had played
- AlphaGo Zero was given the rules of Go and nothing else. No human games, no opening book, no advice. It played against itself, starting from random moves.
- In three days it surpassed the version that had beaten the world champion. Along the way it rediscovered centuries of human opening theory, and then discarded parts of it as inferior.
- It learned from a single number after each game: won or lost. That is a completely different kind of learning from being shown a million labelled photographs.
- This lesson is the three paradigms of machine learning, what data each needs, and how a network's weights are actually changed by backpropagation 反向传播.
Supervised learning
- Supervised learning 监督学习 trains on data that carries labels: photographs tagged "cat" or "dog", emails marked spam or not, houses with their sale prices.
- The model learns the mapping from input to label, and is then given inputs it has never seen and asked to produce the label.
- It splits into classification 分类, where the output is a category, and regression, where the output is a number.
- Its cost is the labels: someone must produce them, which for a million images is a great deal of human work.
Labelled examples in, a model out, then unlabelled inputs
Supervised learning uses:
Supervised learning trains on labelled examples (e.g. images tagged cat/dog).
Unsupervised learning
- Unsupervised learning 无监督学习 is given data with no labels and asked to find structure in it.
- The commonest use is clustering: grouping into clusters 聚类 of similar items, such as customers who buy similar things, without anyone deciding in advance what the groups should be.
- That is its strength and its weakness: it can discover a grouping nobody suspected, and it cannot tell you what the grouping means.
Unsupervised learning is used to:
With no labels, unsupervised learning discovers patterns/clusters in the data.
Reinforcement learning
- Reinforcement learning 强化学习 has an agent 智能体 acting in an environment. Each action changes the state and returns a reward 奖励, which may be zero for a long time.
- The agent learns a policy 策略, a strategy for choosing actions, that maximises its total reward over time. It learns by trial and error, with no labelled examples at all.
- It suits problems that are a sequence of decisions where the right move is only revealed by the eventual outcome: games, robot control, self-driving cars. That is exactly the AlphaGo Zero case.
In reinforcement learning, the agent learns by:
The agent tries actions, receives rewards, and learns a policy that maximises long-term reward.
In reinforcement learning the agent learns a ____ that maximises its total reward over time.
It learns by trial and error from rewards, with no labelled examples at all, which is why it suits sequences of decisions.
Worked example: choose the paradigm
- Ten thousand medical scans, each labelled by a doctor as showing a tumour or not, are used to train a system to flag new scans. Supervised: the data has labels and the task is to learn input to label, specifically a classification.
- A shop wants to know whether its customers fall into natural groups, without deciding the groups in advance. Unsupervised: no labels exist, and the task is to find structure, specifically clustering.
- A robot arm must learn to place components, judged only by whether each attempt succeeded. Reinforcement: an agent, a sequence of actions, and a reward rather than a label.
- Name the paradigm, then justify from the data: labelled, unlabelled, or a reward signal.
AI learning type lab
Classify AI examples by the type of learning or concern involved.
Match each ML paradigm to its data.
Supervised = labels; unsupervised = no labels; reinforcement = reward feedback.
Match each situation to the machine-learning paradigm it needs.
Labels, no labels, or a reward. The data decides the paradigm, not the difficulty of the task.
Training by backpropagation
- Training means adjusting the weights so the network's outputs match the targets. The method is backpropagation with gradient descent 梯度下降.
- Forward pass: feed an input through the network and get its output. Compute the error using a loss function 损失函数, which measures how far the output is from the target.
- Backward pass: propagate that error backwards through the layers to find each weight's gradient, that is, how much that weight contributed to the error.
- Update: change each weight by a small step against its gradient. The step size is the learning rate.
Downhill, one small step at a time
Backpropagation trains a network by:
The error flows from the output back through the network (chain rule) so every weight's gradient is found, then weights step downhill.
Put one round of backpropagation training in order.
Forward, error, backward, update, repeated over many epochs until the error stops falling.
Epochs, and why the learning rate matters
- One pass through the whole training set is an epoch 训练轮次. Training repeats for many epochs until the error stops falling.
- Too large a learning rate overshoots the minimum and the error jumps about; too small and training takes far longer than it needs to.
- After training, using the model is only a forward pass, which is why a trained network answers instantly even though training took days.
In gradient descent the learning rate sets how big a step each weight update takes — too large overshoots the minimum, too small makes training slow.
Backpropagation finds each weight's gradient; the learning rate scales how far down that gradient the weight moves.
Which statements about training are correct? Select all that apply.
The learning rate is the size of each weight update. That is why training can take days while a prediction takes milliseconds.
Marks that slip away
- Justify a paradigm from the data: labelled means supervised, unlabelled means unsupervised, a reward signal means reinforcement.
- Reinforcement learning has no labels. Calling its reward a label is the classic confusion.
- Backpropagation is forward, error, backward, update, repeated. Naming only "it adjusts the weights" is half an answer.
- The learning rate is the size of each step, not the speed of training or the number of epochs.
You've got it
- supervised learns input to label from labelled data, for classification or regression · unsupervised finds structure such as clusters in unlabelled data · reinforcement has an agent learning a policy from rewards by trial and error
- choose the paradigm from what the data provides, not from the task's difficulty
- backpropagation: a forward pass, an error from the loss function, a backward pass giving each weight's gradient, then an update of size set by the learning rate
- training repeats for many epochs; using the trained model is one forward pass