Backpropagation calculus | Deep Learning Chapter 4
Channel:3Blue1Brown · Length:10:17 · Published:2017-11-03 · Views:4034240
Summary
3Blue1Brown's chapter 4 on deep learning gives a more formal, symbol-based account of backpropagation calculus, connecting the intuition from the previous chapter to the math found in textbooks and code.
Why HN recommends it
Recommended as an accessible, visual way to understand backpropagation and neural networks.
3Blue1Brown which explains math in very accessible way — HN, 2017
Machine learning & deep learning · Recreational & visual math
Subtitles (excerpt)
[Submit subtitle corrections at criblate.com] The hard assumption here is that you've watched part 3, giving an intuitive walkthrough of the backpropagation algorithm. Here we get a little more formal and dive into the relevant calculus. It's normal for this to be at least a little confusing, so the mantra to regularly pause and ponder certainly applies as much here as anywhere else. Our main goal is to show how people in machine learning commonly think about the chain rule from calculus in the context of networks, which has a different feel from how most introductory calculus courses approach the subject. For those of you uncomfortable with the relevant calculus, I do have a whole series on the topic. Let's start off with an extremely simple network, one where each layer has a single neuron in it. This network is determined by three weights and three biases, and our goal is to understand how sensitive the cost function is to these variables. That way, we know which adjustments to those terms will cause the most efficient decrease to the cost function. And we're just going to focus on the connection between the last two neurons. Let's label the activation of that last neuron with a superscript L,
On Hacker News
I'm just an enthusiast. With that said, I think the author's first image image really captures the core of forward propagation and back propagation. For people who want to get more of a feel on autodiffing, I find these 2 sources amazing [1, 2]. They don't use the word autodiffing, but looking at it, I can see it's the same thing. I once made an attempt at animating feed forward propagation [3]. … — melvinroest, 2020-06
This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [LINK] [LINK] (Original video) [VIDEO] — nouveaux, 2017-11
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumble… — hal9000xp, 2017-11
It's not satire. If you have some function with several inputs, the Jacobian is a matrix of all the partial derivatives of this function with respect to each input variable[1]. Since derivatives give the slope of a function, if you think about your function as being like a bumpy surface with the height at each point being the output, this matrix tells you which way (and how far) to change any inp… — seanhunter, 2024-10
Related videos
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- Backpropagation, intuitively | Deep Learning Chapter 3 · 3Blue1Brown · 12:47
- Gradient descent, how neural networks learn | Deep Learning Chapter 2 · 3Blue1Brown · 20:33
- Vectors | Chapter 1, Essence of linear algebra · 3Blue1Brown · 9:51
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- But what is the Fourier Transform? A visual introduction. · 3Blue1Brown · 19:41
- The Beauty of Bézier Curves · Freya Holmér · 24:25
- The Continuity of Splines · Freya Holmér · 1:13:50
- Jim Simons (full length interview) - Numberphile · Numberphile2 · 1:00:43
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- But how does bitcoin actually work? · 3Blue1Brown · 25:15
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- Conway's Game Of Life in APL · Dyalog · 7:47
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- MarI/O - Machine Learning for Video Games · SethBling · 5:57