Attention Is All You Need

Channel:Yannic Kilcher · Length:27:06 · Published:2017-11-28 · Views:817445

Watch on YouTube

Summary

Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.

Why HN recommends it

HN recommends it as an accessible, clear walkthrough of the transformer paper for people with some AI background.

The Yannic kilcher review is quite good. — HN, 2023

Machine learning & deep learning · LLMs & generative AI

Subtitles (excerpt)

hi there today we're looking at attention is all you need by Google just to declare I don't work for Google just because we've been looking at Google papers lately but it's just an interesting paper and we're gonna see what's the deal with it so basically what the authors are saying is we should kind of get away from basically onions so traditionally what you would do and these authors particular interested in NLP natural language processing so traditionally when you had like a language task the cat eats the mouse and you would like to translate this to say any other language like let's say German or whatever what you would do is you would try to encode this sentence into a representation and then decode it again so somehow somehow this sentence needs to all go into say one vector and then this one vector needs to somehow be transformed into the target language so these are tradition called sack to sack tasks and they have been solved so far using recurrent neural networks you might know the Alice TN networks that are very popular for these tasks what basically happens in an RNN is that you go over the say source sentence here one by one here you take the word the you kind of encode it maybe with a word vector if you know that is so

On Hacker News

YouTube channel that explains the paper in detail: [VIDEO] And subsequent follow-ups (ROME, editing transformer arch): [LINK] I find the channel amazing explaining super complex topics in simple enough terms for people who have some background in AI. — bfung, 2023-01
The Yannic kilcher review is quite good. [VIDEO] I can't ELI5 but I can ELI-junior-dev. Tl;dw: Transformers work by basically being a differentiable lookup/hash table. First your input is tokenized and (N) tokens (this constitutes the attention frame) are encoded both based on token identity and position in the attention frame. Then there is an NxN matrix that is applied to your attention frame "… — throwawaymaths, 2023-05
For accessibility I recommend Yannic Kilcher video review of "Attention Is All You Need" [VIDEO] Yannic has been making about 62 other transformer paper reviews since. You can find the usual suspects. [LINK] — visarga, 2021-12
Everyone has a different way of approaching. I like video walk-throughs of papers. If you are the same, for the theory bit you may find Yannic Kilcher's videos [0,1] helpful. On implementation side, the best project out there is [2] with good documentation. Also spacy has transformer package [3] (I personally haven't tried it), so maybe it will be easier for you to jump in if you have prior exper… — kanche, 2020-10
> I love these word-salad exaltations of LLMs - people don't even know what they're writing half the time but it sure is provocative! Well FWIW I have done an implementation of Llama LLM (3 8-70b to be specific) in non-python so I do sort of know what I'm talking about. I'm not the originator of the hash table analogy. I got it from here: [VIDEO] Hell, the vectors that are generated are called K/… — throwawaymaths, 2025-05

Related videos