A Hackers' Guide to Language Models

Channel:Jeremy Howard · Length:1:31:13 · Published:2023-09-24 · Views:580620

Watch on YouTube

Summary

Jeremy Howard's 90-minute practical tour of language models. It covers how they work, GPT-4's capabilities and limits, using the OpenAI API, code writing, data analysis and fine-tuning.

Why HN recommends it

Repeatedly named the best practical tutorial for using LLMs, with a well-liked fine-tuning section.

highly recommend "Hackers Guide to language models" by Jeremy Howard. 1.5h video packed with practical information — HN, 2023

Machine learning & deep learning · LLMs & generative AI

Subtitles (excerpt)

hi I am Jeremy Howard from fast.ai and this is a hacker's guide to language models when I say a hacker's guide what we're going to be looking at is a code first approach to understanding how to use language models in practice so before we get started we should probably talk about what is a language model I would say that this is going to make more sense if you know the kind of basics of deep learning if you don't I think you'll still get plenty out of it and there'll be plenty of things you can do but if you do have a chance I would recommend checking out course.fast.ai which is a free course and specifically um if you could at least kind of watch if not work through the first five lessons that would get you to a point where you understand all the basic fundamentals of deep learning that will make this this lesson tutorial make even more sense maybe I shouldn't call this a tutorial it's more of a quick run through so I've got to try to run through all the basic ideas of language models how to use them both open source ones and open AI based ones and it's all going to be based using Code as much as possible um so let's start by talking about what a language model is and so as you might have heard before a language model is

On Hacker News

I understand the sadness around not understanding it, it's fucking hard. however, there are more and more resources online getting published for how to get started understanding it that help with understanding the math at an abstraction that helps with learning how to build with it. I would strongly strongly strongly recommend starting with karpathy's from 0 to hero neural networks youtube course… — mangoman, 2024-02
An interesting solution to the blind spot error (taken directly from Jeremy Howard's amazing guide to language models - [VIDEO] ) is to erase the chat history and try again. Once GPT has made an error (or as the author of this article says, the early layers have irreversibly pruned some important data), it will very often start to be even more wrong. — dojitza1, 2023-11
If you're looking for a practical guide on how to use LLMs, highly recommend "Hackers Guide to language models" by Jeremy Howard. 1.5h video packed with practical information: [VIDEO] — tarruda, 2023-11
Thanks a lot for this video, best LLM usage tutorial I've seen so far. At [VIDEO] when talking about valid use cases for a local model vs GPT4 is: "You might want to create your own model that's particularly good at solving the kinds of problems that you need to solve using fine tuning, and these are all things that you absolutely can get better than GPT4 performance". In regards to this, there's… — tarruda, 2023-09
It sounds like you want more broad stuff, not necessarily learning how to train models. More like learning to use them and how they work. [LINK] and Hacker's Guide to LLMs by Jeremy from Fast.ai - [VIDEO] State of GPT by Karpathy - [LINK] LLMs by 3b1b - [LINK] Visualizing transformers by 3b1b - [LINK] How ChatGPT trained - [LINK] AI in a nutshell - [LINK] How Carlini uses LLMs - [LINK] For stayin… — tikkun, 2024-11

Related videos