Gradient descent, how neural networks learn | Deep Learning Chapter 2
Channel:3Blue1Brown · Length:20:33 · Published:2017-10-16 · Views:9795452
Summary
3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.
Why HN recommends it
HN recommends the series as a very accessible way to understand the math behind machine learning and LLMs.
3Blue1Brown which explains math in very accessible way — HN, 2017
Machine learning & deep learning
Subtitles (excerpt)
Last video I laid out the structure of a neural network. I'll give a quick recap here so that it's fresh in our minds, and then I have two main goals for this video. The first is to introduce the idea of gradient descent, which underlies not only how neural networks learn, but how a lot of other machine learning works as well. Then after that we'll dig in a little more into how this particular network performs, and what those hidden layers of neurons end up looking for. As a reminder, our goal here is the classic example of handwritten digit recognition, the hello world of neural networks. These digits are rendered on a 28x28 pixel grid, each pixel with some grayscale value between 0 and 1. Those are what determine the activations of 784 neurons in the input layer of the network. And then the activation for each neuron in the following layers is based on a weighted sum of all the activations in the previous layer, plus some special number called a bias. Then you compose that sum with some other function, like the sigmoid squishification, or a relu, the way I walked through last video. In total, given the somewhat arbitrary choice of two hidden layers with 16 neurons each, the network has about 13,000 weights and biases that we can adjust,
On Hacker News
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumble… — hal9000xp, 2017-11
> when an LLM tells us that the sun rises in the east, it's not because it truly understands astronomy or the solar system. Rather, it has seen the phrase "sun rises in the east" repeated so many times in its training data that this becomes the highest probability answer. It's less about understanding and more about pattern recognition. This is not too different from most human person's experienc… — superkuh, 2024-10
If you want an extensive answer, I recommend the neural network playlist on 3Blue1Brown: [LINK] For a quick answer, watch this segment: [VIDEO] — miklosme, 2020-05
This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [VIDEO] [LINK] (Original video) [LINK] — nouveaux, 2017-11
A bit of a side-note, but I think it is an interesting piece of marketing that Amplify Partners decided to sponsor[1] the previous video in this series. I wonder (and hope) we'll see more VCs sponsoring open educational content relevant to their focus. [1] [VIDEO] — afarrell, 2017-11
Related videos
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- Backpropagation, intuitively | Deep Learning Chapter 3 · 3Blue1Brown · 12:47
- Backpropagation calculus | Deep Learning Chapter 4 · 3Blue1Brown · 10:17
- But what is the Fourier Transform? A visual introduction. · 3Blue1Brown · 19:41
- How does Computer Memory Work? 💻🛠 · Branch Education · 35:32
- Attention in transformers, step-by-step | Deep Learning Chapter 6 · 3Blue1Brown · 26:09
- AT&T Archives: Similiarities of Wave Behavior (Bonus Edition) · AT&T Tech Channel · 28:02
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- MarI/O - Machine Learning for Video Games · SethBling · 5:57
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- Computer program that learns to play classic NES games · suckerpinch · 16:17
- RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning · Google DeepMind · 1:28:12