Backpropagation, intuitively | Deep Learning Chapter 3
Channel:3Blue1Brown · Length:12:47 · Published:2017-11-03 · Views:6451394
Summary
3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.
Why HN recommends it
Recommended as the most accessible introduction to backprop for people who struggle with the pure math.
I found the explainations by 3Brown1Blue to be very intuitive when it comes to neural networks — HN, 2018
Machine learning & deep learning
Subtitles (excerpt)
Here, we tackle backpropagation, the core algorithm behind how neural networks learn. After a quick recap for where we are, the first thing I'll do is an intuitive walkthrough for what the algorithm is actually doing, without any reference to the formulas. Then, for those of you who do want to dive into the math, the next video goes into the calculus underlying all this. If you watched the last two videos, or if you're just jumping in with the appropriate background, you know what a neural network is, and how it feeds forward information. Here, we're doing the classic example of recognizing handwritten digits whose pixel values get fed into the first layer of the network with 784 neurons, and I've been showing a network with two hidden layers having just 16 neurons each, and an output layer of 10 neurons, indicating which digit the network is choosing as its answer. I'm also expecting you to understand gradient descent, as described in the last video, and how what we mean by learning is that we want to find which weights and biases minimize a certain cost function. As a quick reminder, for the cost of a single training example, you take the output the network gives, along with the output you wanted it to give,
On Hacker News
While we're at this, I found the explainations by 3Brown1Blue to be very intuitive when it comes to neural networks, especially for folks who're new and don't necessarily grasp concepts when explained primarily through math. What is backpropagation really doing? [VIDEO] His other videos on this topic are just as good. — pastelsky, 2018-06
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumble… — hal9000xp, 2017-11
This has a lot of words, but not enough content. I recommend 3Blue1Brown's neural net videos instead: [VIDEO] — sxp, 2021-04
This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [LINK] [VIDEO] (Original video) [LINK] — nouveaux, 2017-11
It's not satire. If you have some function with several inputs, the Jacobian is a matrix of all the partial derivatives of this function with respect to each input variable[1]. Since derivatives give the slope of a function, if you think about your function as being like a bumpy surface with the height at each point being the output, this matrix tells you which way (and how far) to change any inp… — seanhunter, 2024-10
Related videos
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- Gradient descent, how neural networks learn | Deep Learning Chapter 2 · 3Blue1Brown · 20:33
- Backpropagation calculus | Deep Learning Chapter 4 · 3Blue1Brown · 10:17
- Painting a Character with Maths · Inigo Quilez · 24:46
- James Burke Connections, Ep. 4 "Faith in Numbers" · DRS_Education · 48:41
- James Burke Connections, Ep. 1 "The Trigger Effect" · DRS_Education · 49:22
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- MarI/O - Machine Learning for Video Games · SethBling · 5:57
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- Computer program that learns to play classic NES games · suckerpinch · 16:17
- RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning · Google DeepMind · 1:28:12
- Let's build GPT: from scratch, in code, spelled out. · Andrej Karpathy · 1:56:20