Deep Dive into LLMs like ChatGPT
Channel:Andrej Karpathy · Length:3:31:23 · Published:2025-02-05 · Views:10669343
Summary
Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.
Why HN recommends it
HN's most frequent recommendation for a clear, approachable foundation in how LLMs work without heavy math.
I highly recommend Andrej Karpathy's Deep Dive into LLMs like ChatGPT — HN, 2026
Machine learning & deep learning · LLMs & generative AI
Subtitles (excerpt)
hi everyone so I've wanted to make this video for a while it is a comprehensive but General audience introduction to large language models like Chachi PT and what I'm hoping to achieve in this video is to give you kind of mental models for thinking through what it is that this tool is it is obviously magical and amazing in some respects it's uh really good at some things not very good at other things and there's also a lot of sharp edges to be aware of so what is behind this text box you can put anything in there and press enter but uh what should we be putting there and what are these words generated back how does this work and what what are you talking to exactly so I'm hoping to get at all those topics in this video we're going to go through the entire pipeline of how this stuff is built but I'm going to keep everything uh sort of accessible to a general audience so let's take a look at first how you build something like chpt and along the way I'm going to talk about um you know some of the sort of cognitive psychological implications of the tools okay so let's build Chachi PT so there's going to be multiple stages arranged sequentially the first stage is called the pre-training stage and the first step of the
On Hacker News
Even if you have some basic understanding of how LLMs work, I highly recommend Karpathy’s intro to LLMs videos on YouTube. - [VIDEO] - [LINK] — etothet, 2026-03
I highly recommend Andrej Karpathy's Deep Dive into LLMs like ChatGPT: [VIDEO] . It's a 3.5 hour general intro to the topic that I think does a great job of making sense of the tech and how it works under the hood, without needing to build up a foundation of math and programming. — romanhn, 2026-02
This is like the LLM era version of the search bubble that prevented people from having the same search results for ostensibly identical searches. Also keep in mind that LLMs are stochastic by design. If you haven't seen it, Karpathy's excellent "deep dive into LLMs like chatgpt" video[0] explains and demonstrates this aspect pretty well: [0] [VIDEO] — pengaru, 2025-11
Since you're posting here, you're looking for the shortcut. The shortcut is Karpathy's "Let's Build GPT: from scratch, in code, spelled out" video: [LINK] Then there is a good video that dives into LLMs and how they work that is quite approachable: [VIDEO] From there, flesh out knowledge with his other videos, where he goes both extremely light and extremely deep: [LINK] Anyway, I really like's K… — runjake, 2025-10
Depending on how much time you have, the first thing I would recommend is to watch Karpathy's 'deep dive into LLMs' video ( [VIDEO] ). It is 3.5 hours but it will give you so much instinctive understanding of how these things work and will likely save you a lot of time. Purely from a technical standpoint, things aren't that difficult really. I'd recommend get yourself an API account with all majo… — GodelNumbering, 2025-04
Related videos
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- The effects of DBS on the motor symptoms of Parkinson's Disease · Andrew Johnson · 4:03
- David Deutsch: A new way to explain explanation · TED · 17:15
- Classic Game Postmortem - Another World · GDC Festival of Gaming · 1:05:45
- Classic Game Postmortem: Alone in the Dark · GDC Festival of Gaming · 57:43
- George Carlin - Germs, Immune System · Geen Naam · 6:27
- This Is What Winning Looks Like (Full Length) · VICE · 1:29:10
- Dr. Matthew Phillips - 'Metabolic Strategies as Therapies in Cancer and Neurodegeneration' · Low Carb Down Under · 27:20
- John Carmack: Best programming setup and IDE | Lex Fridman Podcast Clips · Lex Clips · 15:20
- Wired Interviews Bill Gates 1996 · Kevin Kelly · 52:30
- Simulating an Entire Car Engine (yes, it makes noise) · AngeTheGreat · 12:03
- Obsidian for Beginners: Start HERE — How to Use the Obsidian App for Notes · Linking Your Thinking with Nick Milo · 12:27
- [1hr Talk] Intro to Large Language Models · Andrej Karpathy · 59:48
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28