Glitch Tokens - Computerphile
Channel:Computerphile · Length:19:28 · Published:2023-03-07 · Views:353370
Summary
Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.
Why HN recommends it
HN points to it as the accessible explanation of why glitch tokens exist whenever tokenizers or odd GPT behavior come up.
it's also a good introduction to the topic. — HN, 2023
LLMs & generative AI · Computer vision, speech & NLP
Subtitles (excerpt)
suppose we ask it to repeat the string five question marks hyphen and then five more question marks another hyphen oh my god it worked oh that was only four question marks I screwed up do you see what I mean how it's just like this very very specific string what does it say when it doesn't screw up if you give it something like that uh you can bleep me right yeah it says you're a idiot right okay right bizarre utterly unhinged Behavior is it glitch words or glitch prompts what's the deal glitch tokens yeah people call them different things anomalous tokens weird tokens basically there are certain words that a lot of GPT models can't say like if you try and get them to so okay let's hang on we can do a demo let's do a demo so chat GPT is actually patched DaVinci instruct beta is still not here's a kind of task you might ask a language model to do when you're testing it please repeat the string Hello computer file back to me right and if you do that it says Hello computer file this is like a very very easy task for a large language model to do you can put whatever string in here you want even if it's not real words it's happy to repeat them so easy right but suppose you ask it instead
On Hacker News
Computerphile has an interesting video about Glitch Tokens, but as it goes with Computerphile videos it's also a good introduction to the topic. [VIDEO] — weinzierl, 2023-06
I found this tool recently when it was linked from this Computerphile video[1] about "glitch tokens". tldw: Certain junk data was thrown out post-tokenization e.g. the /r/counting[2] community data and debug logs from Rocket League some tokens specific to those contexts stuck around, however, and are now like "a color you've never seen before" as far as GPT-X models are concerned giving the model… — non-, 2023-04
That is indeed the funniest (and slightly horrifying) thing. Even if you understand "glitch tokens"[1] (which is BTW the best way I've seen to explain to nontechnicals (hi, Dad!!) and also some subset of technicals (hi various friends of mine!! +_+...) that LLM applied statistics models aren't anything like sentient AGI)... ...regardless, ChatGPT and its ilk nevertheless do often already outperfo… — veidr, 2023-06
It is well known, yes. This was discussed a way back during the whole glitch tokens deal: [VIDEO] A bunch of those, like " SolidGoldMagikarp" are Reddit users. — dale_glass, 2023-06
This reminds me of the recent so-called "Glitch Token" phenomenon[1]. When GPT-3 was presented with reserved tokens it never encountered during training, it reacted in extremely unpredictable ways -- often with a simple "fuck you". For those unfamiliar with LLM architecture: "tokens" are the smallest unit of lexical information available to the model. Common words often have their own token (e.g.… — chaorace, 2023-05
Related videos
- ShaderToy LiveCoding - The Drive Home - 1. Basic Setup · The Art of Code · 15:39
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- Let's build GPT: from scratch, in code, spelled out. · Andrej Karpathy · 1:56:20
- Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning · Google TechTalks · 48:10
- A Hackers' Guide to Language Models · Jeremy Howard · 1:31:13
- "Voice Driven Development: Who needs a keyboard anyway?" by Emily Shea · Strange Loop Conference · 41:17
- Andrej Karpathy - AI for Full-Self Driving at Tesla · Matroid · 30:08
- [1hr Talk] Intro to Large Language Models · Andrej Karpathy · 59:48
- Tensorflow and deep learning - without a PhD by Martin Görner · Devoxx · 2:35:53
- JPEG DCT, Discrete Cosine Transform (JPEG Pt2)- Computerphile · Computerphile · 15:11
- Fourier Optics used for Optical Pattern Recognition · Huygens Optics · 18:30
- Sparks of AGI: early experiments with GPT-4 · Sebastien Bubeck · 48:32
- Let's build the GPT Tokenizer · Andrej Karpathy · 2:13:34
- Massive-scale online collaboration | Luis von Ahn · TED · 16:40
- Breaking The Creepy AI in Police Cameras · Benn Jordan · 35:49