Neural network math

Lecture

What you’ll learn

• A neuron is weight × input + bias, the same as the equation of a line
• Activation functions and why networks need them
• Loss: a single number for “how wrong are we?”
• Gradient descent: walking downhill on the loss
• Backpropagation as the chain rule, step by step
• Why the learning rate can be too small or too big

Lecture outline

1. Lines, slopes and predictions
2. From one neuron to a layer
3. Measuring mistakes with a loss function
4. Rolling downhill: gradient descent
5. Backpropagation without fear
6. Putting it together: a tiny network that learns