Backpropagation, intuitively | Deep Learning Chapter 3

Channel:3Blue1Brown · Length:12:47 · Published:2017-11-03 · Views:6451394

Watch on YouTube

Summary

3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.

Why HN recommends it

Recommended as the most accessible introduction to backprop for people who struggle with the pure math.

I found the explainations by 3Brown1Blue to be very intuitive when it comes to neural networks — HN, 2018

Machine learning & deep learning

Subtitles (excerpt)

Here, we tackle backpropagation, the core algorithm behind how neural networks learn. After a quick recap for where we are, the first thing I'll do is an intuitive walkthrough for what the algorithm is actually doing, without any reference to the formulas. Then, for those of you who do want to dive into the math, the next video goes into the calculus underlying all this. If you watched the last two videos, or if you're just jumping in with the appropriate background, you know what a neural network is, and how it feeds forward information. Here, we're doing the classic example of recognizing handwritten digits whose pixel values get fed into the first layer of the network with 784 neurons, and I've been showing a network with two hidden layers having just 16 neurons each, and an output layer of 10 neurons, indicating which digit the network is choosing as its answer. I'm also expecting you to understand gradient descent, as described in the last video, and how what we mean by learning is that we want to find which weights and biases minimize a certain cost function. As a quick reminder, for the cost of a single training example, you take the output the network gives, along with the output you wanted it to give,

On Hacker News

While we're at this, I found the explainations by 3Brown1Blue to be very intuitive when it comes to neural networks, especially for folks who're new and don't necessarily grasp concepts when explained primarily through math. What is backpropagation really doing? [VIDEO] His other videos on this topic are just as good. — pastelsky, 2018-06
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumble… — hal9000xp, 2017-11
This has a lot of words, but not enough content. I recommend 3Blue1Brown's neural net videos instead: [VIDEO] — sxp, 2021-04
This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [LINK] [VIDEO] (Original video) [LINK] — nouveaux, 2017-11
It's not satire. If you have some function with several inputs, the Jacobian is a matrix of all the partial derivatives of this function with respect to each input variable[1]. Since derivatives give the slope of a function, if you think about your function as being like a bumpy surface with the height at each point being the output, this matrix tells you which way (and how far) to change any inp… — seanhunter, 2024-10

Related videos