AI Language Models & Transformers - Computerphile

Channel:Computerphile · Length:20:39 · Published:2019-06-26 · Views:354480

Watch on YouTube

Summary

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

Why HN recommends it

Pointed to as an accessible early explanation of transformers and attention for newcomers.

You can check-out his other videos on NLP as well they are all excellent. — HN, 2023

Machine learning & deep learning · LLMs & generative AI · Computer vision, speech & NLP

Subtitles (excerpt)

cSo I wanted to make a video about GPT - 2 Because it's been in the news recently this very powerful language model from open AI and I thought it would make sense to start by just doing a video about transformers and language models in general because GPT 2 is a very large Language model implemented as a transformer, but you have a previous video about generating YouTube comments, which is the same kind of task, right? That's a language modeling task from language processing to generate new samples for cooling of the most complex or magnetic Consistent brackets like a computer to expect found in creating organizations I believe that video was made October 2017 and this paper came out December 2017, which has kind of Revolutionized the way that people carry out that kind of task. That's not the GPT - 2 that's something before that, right? That's the transformer, which is a new realm. Yeah relatively new architecture for neural networks, that can do actually all kinds of tasks, but they're especially good at this kind of language modeling task a language model is a probability distribution over like sequences of Tokens or symbols or words or whatever in a language? So for any given like sequence of tokens, it can tell you how likely that is

On Hacker News

If you want to get a better understanding of this I recommend playing around in the "chat playgrounds" on some of the engines. The Google one allows for some free use before you have to pay for tokens. (Usually you can buy $5 worth of tokens as a minimum and that will give you more than you can use up with manual requests.) [LINK] This UI allows you to alter the system prompt (which is usually hi… — mhast, 2025-02
Those Computerphile videos[0] by Rob Miles helped me understand transformers. He specifically references the "Attention is all you need" paper. And for a deeper dive, Andrej Kharpaty has this hands-on video[1] where he builds a transformer from scratch. You can check-out his other videos on NLP as well they are all excellent. [0] [VIDEO] , [LINK] [1] [LINK] — FranklinMaillot, 2023-05
This video by Computerphile is a great overview of transformers and how we got to this point [0]. Basically the networks we used before, recurrent neural networks, "forgot" prior information so they're not good at long tasks. The transformer architecture however does not forget (or at least as easily). [0] [VIDEO] — cercatrova, 2022-08

Related videos