AI, graphs and neural networks
| English | Chinese | Pinyin |
|---|---|---|
| machine learning | 机器学习 | jī qì xué xí |
| deep learning | 深度学习 | shēn dù xué xí |
| graph | 图 | tú |
| neural networks | 神经网络 | shén jīng wǎng luò |
| nodes | 节点 | jié diǎn |
| edges | 边 | biān |
| minimax | 极小化极大 | jí xiǎo huà jí dà |
| weight | 权重 | quán zhòng |
| activation function | 激活函数 | jī huó hán shù |
| hidden layers | 隐藏层 | yǐn cáng céng |
The program that learned to see without being told what to look for
- In 2012 a team entered an image-recognition competition with a program that had never been given a single rule about what a cat looks like. It had been shown a million labelled photographs, and it worked out the rules itself.
- It halved the error rate of every hand-programmed entry. Within three years the entire field had abandoned writing rules.
- That reversal is the subject of this topic: not "how do we tell a computer what a cat is", but "how does a computer work that out from examples".
- This lesson is what machine learning 机器学习 and deep learning 深度学习 mean, how a graph 图 turns AI problems into search, and what an artificial neural network actually computes.
AI, machine learning and deep learning
- Artificial intelligence is the broad goal: systems performing tasks that normally need human intelligence, such as vision, speech, translation and driving.
- Machine learning is the approach that dominates it: algorithms that learn patterns from data rather than being programmed step by step.
- Deep learning is machine learning using neural networks 神经网络 with many layers. It has led the field since the 2010s because it learns straight from raw data, pixels or audio or text, with no hand-designed features.
- They nest: deep learning is part of machine learning, which is part of AI.
Three circles, not three rivals
Machine learning means an algorithm that:
ML learns from data; deep learning is ML using multi-layer neural networks.
How do AI, machine learning and deep learning relate?
AI is the goal, machine learning is the dominant approach to it, and deep learning is machine learning with many-layered networks.
Graphs, and why so much AI is search
- A graph is a set of nodes 节点 joined by edges 边. Nodes are states, places or concepts; edges are moves, roads or relationships.
- Pathfinding: a road network is a graph, and finding a route is a graph search, using Dijkstra's algorithm or A*.
- Game playing: every board position is a node and every legal move an edge, so the game is a tree that minimax 极小化极大 searches, assuming each player takes their best option.
- Knowledge representation: a semantic network holds concepts as nodes and relationships as edges. Breadth-first and depth-first search are the standard tools for exploring any of these.
In AI, a graph consists of:
Graphs model states/places as nodes and moves/relationships as edges — used for pathfinding, game trees and knowledge.
Worked example: model a problem as a graph
- A delivery company wants the shortest route between two towns. Describe how this is represented as a graph.
- Each town is a node; each road is an edge joining two nodes; the edge carries a weight, the distance or the travel time.
- The problem is then "find the path between the two nodes with the smallest total weight", which Dijkstra's algorithm solves.
- Say what the nodes are, what the edges are, and what the weights mean. An answer that only says "it is a graph" earns nothing.
A road network is modelled as a graph to find the shortest route. Which statements are correct? Select all that apply.
Places are nodes and connections are edges. Saying what the weights represent is the third mark.
The artificial neuron
- An artificial neuron takes several inputs. It multiplies each input by a weight 权重, adds them together with a bias, then passes the total through a non-linear activation function 激活函数 to produce its output.
- The weights are what the network learns. The activation function is what makes the network able to represent something other than a straight line: without it, any number of layers would collapse into a single linear formula.
Multiply, add, then bend
Put the steps inside one artificial neuron in order.
Multiply, sum, activate, pass on. Without the non-linear activation, any depth of network would collapse into one linear formula.
Layers, and what "deep" means
- Neurons are arranged in layers. The input layer receives the data. One or more hidden layers 隐藏层 learn internal patterns. The output layer produces the answer, a class or a number.
- A network is called deep when it has many hidden layers, and training such a network is deep learning.
- Why the hidden layers matter: in an image network the early ones learn edges, the middle ones shapes, the later ones objects. Nobody designed that hierarchy; it is what training produced.
Input, hidden, output, and the learning is in the weights between them
Tap the parts of a neural network
Explore the layers. Data flows left to right: the input layer takes the features, the hidden layers learn patterns, and the output layer gives the answer — with every connection carrying a weight that training adjusts.
Match each part of a neural network to its role.
Features enter the input layer, hidden layers learn patterns, the output layer answers; learning = tuning the weights.
A neural network is called "deep" when it has many hidden layers, and each neuron takes a weighted sum of its inputs (plus a bias) before applying an activation function.
Stacking many hidden layers lets the network learn richer patterns — training such a network is deep learning.
A neural network is called "deep" when it has:
Many hidden layers make a deep neural network; training one is deep learning.
During training, the values adjusted inside a neural network are its ____.
The architecture and the activation function are chosen by the designer; what the network learns is stored in the weights.
Worked example: describe how a neuron computes
- Describe what happens inside a single artificial neuron. [3]
- Each input is multiplied by its own weight, and the products are added together along with a bias.
- The resulting sum is passed through an activation function, which is non-linear.
- The result is the neuron's output, which becomes an input to the neurons in the next layer. The weights and bias are the values adjusted during training.
Marks that slip away
- Machine learning learns patterns from data; it is not "programmed step by step" and it is not magic. Say what it learns from.
- "Deep" means many hidden layers, not "very clever" or "very large".
- A graph is nodes and edges, and a graph answer must say what each represents in this problem.
- The weights are what training changes. The activation function and the architecture are chosen by the designer.
You've got it
- AI is the goal, machine learning learns patterns from data instead of following written rules, and deep learning is machine learning with many-layered neural networks
- a graph of nodes and edges turns pathfinding, game playing and knowledge representation into search, using Dijkstra, A*, minimax, breadth-first and depth-first
- an artificial neuron multiplies inputs by weights, adds a bias, and applies a non-linear activation function
- layers run input, hidden, output; deep means many hidden layers, and the learning lives in the weights