Backpropagation calculus | Deep Learning Chapter 4

Channel:3Blue1Brown · Length:10:17 · Published:2017-11-03 · Views:4034240

Watch on YouTube

Summary

3Blue1Brown's chapter 4 on deep learning gives a more formal, symbol-based account of backpropagation calculus, connecting the intuition from the previous chapter to the math found in textbooks and code.

Why HN recommends it

Recommended as an accessible, visual way to understand backpropagation and neural networks.

3Blue1Brown which explains math in very accessible way — HN, 2017

Machine learning & deep learning · Recreational & visual math

Subtitles (excerpt)

[Submit subtitle corrections at criblate.com] The hard assumption here is that you've watched part 3, giving an intuitive walkthrough of the backpropagation algorithm. Here we get a little more formal and dive into the relevant calculus. It's normal for this to be at least a little confusing, so the mantra to regularly pause and ponder certainly applies as much here as anywhere else. Our main goal is to show how people in machine learning commonly think about the chain rule from calculus in the context of networks, which has a different feel from how most introductory calculus courses approach the subject. For those of you uncomfortable with the relevant calculus, I do have a whole series on the topic. Let's start off with an extremely simple network, one where each layer has a single neuron in it. This network is determined by three weights and three biases, and our goal is to understand how sensitive the cost function is to these variables. That way, we know which adjustments to those terms will cause the most efficient decrease to the cost function. And we're just going to focus on the connection between the last two neurons. Let's label the activation of that last neuron with a superscript L,

On Hacker News

I'm just an enthusiast. With that said, I think the author's first image image really captures the core of forward propagation and back propagation. For people who want to get more of a feel on autodiffing, I find these 2 sources amazing [1, 2]. They don't use the word autodiffing, but looking at it, I can see it's the same thing. I once made an attempt at animating feed forward propagation [3]. … — melvinroest, 2020-06
This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [LINK] [LINK] (Original video) [VIDEO] — nouveaux, 2017-11
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumble… — hal9000xp, 2017-11
It's not satire. If you have some function with several inputs, the Jacobian is a matrix of all the partial derivatives of this function with respect to each input variable[1]. Since derivatives give the slope of a function, if you think about your function as being like a bumpy surface with the height at each point being the output, this matrix tells you which way (and how far) to change any inp… — seanhunter, 2024-10

Related videos