The spelled-out intro to neural networks and backpropagation: building micrograd
Channel:Andrej Karpathy · Length:2:25:52 · Published:2022-08-16 · Views:4502046
Summary
Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.
Why HN recommends it
HN treats it as the best starting point for really understanding backprop and neural nets, because Karpathy explains the why with intuition and builds everything in code.
It's especially useful because he explains the why and provides intuitive explanations instead of just talking about the what and how. — HN, 2023
Python, Ruby & scripting languages · Machine learning & deep learning
Subtitles (excerpt)
hello my name is andre and i've been training deep neural networks for a bit more than a decade and in this lecture i'd like to show you what neural network training looks like under the hood so in particular we are going to start with a blank jupiter notebook and by the end of this lecture we will define and train in neural net and you'll get to see everything that goes on under the hood and exactly sort of how that works on an intuitive level now specifically what i would like to do is i would like to take you through building of micrograd now micrograd is this library that i released on github about two years ago but at the time i only uploaded the source code and you'd have to go in by yourself and really figure out how it works so in this lecture i will take you through it step by step and kind of comment on all the pieces of it so what is micrograd and why is it interesting good um micrograd is basically an autograd engine autograd is short for automatic gradient and really what it does is it implements backpropagation now backpropagation is this algorithm that allows you to efficiently evaluate the gradient of some kind of a loss function with respect to the weights of a neural network and what that allows
On Hacker News
If you have 20 hours to spare I highly recommend this youtube playlist from Andrej Karpathy [VIDEO] It starts with the fundamentals of how backpropagation works then advances to building a few simple models and ends with building a GPT-2 clone. It won't taech you everything about AI models but it gives you a solid foundation for branching out. — jwitthuhn, 2025-12
This blog post analyzes the code with an emphasis on the small details and ideas explored in Andrej Karpathy's awesome micrograd lecture. Link to the original lecture: [VIDEO] I'd love to hear any thougts/reviews about the blog post. — omrishn, 2025-11
For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers [LINK] and so on. And of course Yannic Kilcher[4], and also listening in on the pape… — jackdoe, 2025-06
In the same vein, Karpathy's video series "Neural Networks from zero to hero"[0] touches upon a lot of this and intuitions as well. One of the best introductory series (even if you ignore the neural net part of it) and brushes on gradients, differentiation and what it means intuitively. [0] [VIDEO] — throwaway7783, 2025-05
Watch Karpathy's recent lectures. They're gold. Start here[1] with micrograd[2]. It doesn't use linear algebra/matrices to start, but the principles are the same. The matrix multiplication is how the weights of the connections between neurons and the input values are combined (to form an activation value that then may lead to that neuron "firing" or not, depending on whether it passes some thresh… — paulsmith, 2023-09
Related videos
- Richard Feynman Messenger Lectures at Cornell The Character of Physical Law Part 1 The Law of Gravitation · TalkReelTV · 56:04
- The Biggest Ideas in the Universe: Introduction · Sean Carroll · 6:33
- Lecture 1 | Modern Physics: Classical Mechanics (Stanford) · Stanford · 47:50
- College Lecture Series - Neil Postman - "The Surrender of Culture to Technology" · College of DuPage · 1:25:11
- Lecture 1 | Introduction to Convolutional Neural Networks for Visual Recognition · Stanford University School of Engineering · 57:56
- Preface | MIT Calculus Revisited: Single Variable Calculus · MIT OpenCourseWare · 32:06
- Elon Musk Explains SpaceX's Raptor Engine! · Everyday Astronaut · 41:22
- Jeff Hawkins: Thousand Brains Theory of Intelligence | Lex Fridman Podcast #25 · Lex Fridman · 2:09:42
- Steve Jobs on Virtual Reality (D3, 2005) · Milan Lajtoš · 0:34
- Bitcoin Is Protecting Human Rights Around the World · ReasonTV · 5:38
- Here Comes Another Bubble v1.1 - The Richter Scales · TheRichterScales · 2:45
- Interactive programming Flappy Bird in ClojureScript · Bruce Hauman · 6:22
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- Ruby Conf 12 - Y Not- Adventures in Functional Programming by Jim Weirich · Confreaks · 53:21
- How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework · Видео с мероприятий {speach! · 49:55
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47