機械学習の種類と学習

The program that taught itself an opening no human had played

• AlphaGo Zero was given the rules of Go and nothing else. No human games, no opening book, no advice. It played against itself, starting from random moves.
• In three days it surpassed the version that had beaten the world champion. Along the way it rediscovered centuries of human opening theory, and then discarded parts of it as inferior.
• It learned from a single number after each game: won or lost. That is a completely different kind of learning from being shown a million labelled photographs.
• This lesson is the three paradigms of machine learning, what data each needs, and how a network's weights are actually changed by backpropagation 反向传播.

Supervised learning

• Supervised learning 监督学习 trains on data that carries labels: photographs tagged "cat" or "dog", emails marked spam or not, houses with their sale prices.
• The model learns the mapping from input to label, and is then given inputs it has never seen and asked to produce the label.
• It splits into classification 分类, where the output is a category, and regression, where the output is a number.
• Its cost is the labels: someone must produce them, which for a million images is a great deal of human work.

Labelled examples in, a model out, then unlabelled inputs

Unsupervised learning

• Unsupervised learning 无监督学习 is given data with no labels and asked to find structure in it.
• The commonest use is clustering: grouping into clusters 聚类 of similar items, such as customers who buy similar things, without anyone deciding in advance what the groups should be.
• That is its strength and its weakness: it can discover a grouping nobody suspected, and it cannot tell you what the grouping means.

Reinforcement learning

• Reinforcement learning 强化学习 has an agent 智能体 acting in an environment. Each action changes the state and returns a reward 奖励, which may be zero for a long time.
• The agent learns a policy 策略, a strategy for choosing actions, that maximises its total reward over time. It learns by trial and error, with no labelled examples at all.
• It suits problems that are a sequence of decisions where the right move is only revealed by the eventual outcome: games, robot control, self-driving cars. That is exactly the AlphaGo Zero case.

Worked example: choose the paradigm

Training by backpropagation

• Training means adjusting the weights so the network's outputs match the targets. The method is backpropagation with gradient descent 梯度下降.
• Forward pass: feed an input through the network and get its output. Compute the error using a loss function 损失函数, which measures how far the output is from the target.
• Backward pass: propagate that error backwards through the layers to find each weight's gradient, that is, how much that weight contributed to the error.
• Update: change each weight by a small step against its gradient. The step size is the learning rate.

Downhill, one small step at a time

Epochs, and why the learning rate matters

• One pass through the whole training set is an epoch 训练轮次. Training repeats for many epochs until the error stops falling.
• Too large a learning rate overshoots the minimum and the error jumps about; too small and training takes far longer than it needs to.
• After training, using the model is only a forward pass, which is why a trained network answers instantly even though training took days.

Marks that slip away

You've got it