Machine learning types and training
Introduced| English | Meaning |
|---|---|
| machine learning/məˈʃiːn ˈlɜːnɪŋ/ | Methods that use data to fit models for making predictions or identifying patterns. |
| backpropagation/ˌbækprəpəˈɡeɪʃn/ | Computing how a network's loss depends on its parameters by propagating gradients backwards through its layers. |
| supervised learning/ˈsuːpəvaɪzd ˈlɜːnɪŋ/ | supervised learning |
| classification/ˌklæsɪfɪˈkeɪʃn/ | Arranging items or organisms into groups according to shared features or a defined system. |
| unsupervised learning/ʌnˈsuːpəvaɪzd ˈlɜːnɪŋ/ | unsupervised learning |
| clusters/ˈklʌstəz/ | clusters |
| reinforcement learning/ˌriːɪnˈfɔːsmənt ˈlɜːnɪŋ/ | reinforcement learning |
| agent/ˈeɪdʒənt/ | A system that observes an environment and takes actions within it to pursue an objective. |
| reward/rɪˈwɔːd/ | A feedback signal used in reinforcement learning to evaluate outcomes and guide how an agent learns to act. |
| policy/ˈpɒlɪsi/ | In reinforcement learning, a rule or distribution that determines how an agent selects actions from its observations or state. |
| gradient descent/ˈɡreɪdɪənt dɪˈsent/ | An iterative method that adjusts parameters in the direction opposite the gradient to seek a smaller value of an objective function. |
| loss function/lɒs ˈfʌŋkʃn/ | A function that measures how poorly a model's predictions or outputs meet the training objective. |
| epoch/ˈiːpɒk/ | epoch |
The program that taught itself an opening no human had played
- AlphaGo Zero was given the rules of Go and nothing else. No human games, no opening book, no advice. It played against itself, starting from random moves.
- In three days it surpassed the version that had beaten the world champion. Along the way it rediscovered centuries of human opening theory, and then discarded parts of it as inferior.
- It learned from a single number after each game: won or lost. That is a completely different kind of learning from being shown a million labelled photographs.
- This lesson is the three paradigms of machine learning, what data each needs, and how a network's weights are actually changed by backpropagation 反向传播.
Supervised learning
- Supervised learning 监督学习 trains on data that carries labels: photographs tagged "cat" or "dog", emails marked spam or not, houses with their sale prices.
- The model learns the mapping from input to label, and is then given inputs it has never seen and asked to produce the label.
- It splits into classification 分类, where the output is a category, and regression, where the output is a number.
- Its cost is the labels: someone must produce them, which for a million images is a great deal of human work.
Labelled examples in, a model out, then unlabelled inputs
Supervised learning uses:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Supervised learning trains on labelled examples (e.g. images tagged cat/dog).
Unsupervised learning
- Unsupervised learning 无监督学习 is given data with no labels and asked to find structure in it.
- The commonest use is clustering: grouping into clusters 聚类 of similar items, such as customers who buy similar things, without anyone deciding in advance what the groups should be.
- That is its strength and its weakness: it can discover a grouping nobody suspected, and it cannot tell you what the grouping means.
Unsupervised learning is used to:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
With no labels, unsupervised learning discovers patterns/clusters in the data.
Reinforcement learning
- Reinforcement learning 强化学习 has an agent 智能体 acting in an environment. Each action changes the state and returns a reward 奖励, which may be zero for a long time.
- The agent learns a policy 策略, a strategy for choosing actions, that maximises its total reward over time. It learns by trial and error, with no labelled examples at all.
- It suits problems that are a sequence of decisions where the right move is only revealed by the eventual outcome: games, robot control, self-driving cars. That is exactly the AlphaGo Zero case.
In reinforcement learning, the agent learns by:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
The agent tries actions, receives rewards, and learns a policy that maximises long-term reward.
In reinforcement learning the agent learns a ____ that maximises its total reward over time.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
It learns by trial and error from rewards, with no labelled examples at all, which is why it suits sequences of decisions.
Worked example: choose the paradigm
- Ten thousand medical scans, each labelled by a doctor as showing a tumour or not, are used to train a system to flag new scans. Supervised: the data has labels and the task is to learn input to label, specifically a classification.
- A shop wants to know whether its customers fall into natural groups, without deciding the groups in advance. Unsupervised: no labels exist, and the task is to find structure, specifically clustering.
- A robot arm must learn to place components, judged only by whether each attempt succeeded. Reinforcement: an agent, a sequence of actions, and a reward rather than a label.
- Name the paradigm, then justify from the data: labelled, unlabelled, or a reward signal.
AI learning type lab
Classify AI examples by the type of learning or concern involved.
Match each ML paradigm to its data.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Supervised = labels; unsupervised = no labels; reinforcement = reward feedback.
Match each situation to the machine-learning paradigm it needs.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Labels, no labels, or a reward. The data decides the paradigm, not the difficulty of the task.
Training by backpropagation
- Training means adjusting the weights so the network's outputs match the targets. The method is backpropagation with gradient descent 梯度下降.
- Forward pass: feed an input through the network and get its output. Compute the error using a loss function 损失函数, which measures how far the output is from the target.
- Backward pass: propagate that error backwards through the layers to find each weight's gradient, that is, how much that weight contributed to the error.
- Update: change each weight by a small step against its gradient. The step size is the learning rate.
Downhill, one small step at a time
Backpropagation trains a network by:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
The error flows from the output back through the network (chain rule) so every weight's gradient is found, then weights step downhill.
Put one round of backpropagation training in order.
Difficulty 3/5 · 200 coins + a small bonus on first completion today
Forward, error, backward, update, repeated over many epochs until the error stops falling.
Epochs, and why the learning rate matters
- One pass through the whole training set is an epoch 训练轮次. Training repeats for many epochs until the error stops falling.
- Too large a learning rate overshoots the minimum and the error jumps about; too small and training takes far longer than it needs to.
- After training, using the model is only a forward pass, which is why a trained network answers instantly even though training took days.
In gradient descent the learning rate sets how big a step each weight update takes — too large overshoots the minimum, too small makes training slow.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Backpropagation finds each weight's gradient; the learning rate scales how far down that gradient the weight moves.
Which statements about training are correct? Select all that apply.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
The learning rate is the size of each weight update. That is why training can take days while a prediction takes milliseconds.
Marks that slip away
- Justify a paradigm from the data: labelled means supervised, unlabelled means unsupervised, a reward signal means reinforcement.
- Reinforcement learning has no labels. Calling its reward a label is the classic confusion.
- Backpropagation is forward, error, backward, update, repeated. Naming only "it adjusts the weights" is half an answer.
- The learning rate is the size of each step, not the speed of training or the number of epochs.
You've got it
- supervised learns input to label from labelled data, for classification or regression · unsupervised finds structure such as clusters in unlabelled data · reinforcement has an agent learning a policy from rewards by trial and error
- choose the paradigm from what the data provides, not from the task's difficulty
- backpropagation: a forward pass, an error from the loss function, a backward pass giving each weight's gradient, then an update of size set by the learning rate
- training repeats for many epochs; using the trained model is one forward pass