Glitch Tokens - Computerphile

Channel:Computerphile · Length:19:28 · Published:2023-03-07 · Views:353370

Watch on YouTube

Summary

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

Why HN recommends it

HN points to it as the accessible explanation of why glitch tokens exist whenever tokenizers or odd GPT behavior come up.

it's also a good introduction to the topic. — HN, 2023

LLMs & generative AI · Computer vision, speech & NLP

Subtitles (excerpt)

suppose we ask it to repeat the string five question marks hyphen and then five more question marks another hyphen oh my god it worked oh that was only four question marks I screwed up do you see what I mean how it's just like this very very specific string what does it say when it doesn't screw up if you give it something like that uh you can bleep me right yeah it says you're a idiot right okay right bizarre utterly unhinged Behavior is it glitch words or glitch prompts what's the deal glitch tokens yeah people call them different things anomalous tokens weird tokens basically there are certain words that a lot of GPT models can't say like if you try and get them to so okay let's hang on we can do a demo let's do a demo so chat GPT is actually patched DaVinci instruct beta is still not here's a kind of task you might ask a language model to do when you're testing it please repeat the string Hello computer file back to me right and if you do that it says Hello computer file this is like a very very easy task for a large language model to do you can put whatever string in here you want even if it's not real words it's happy to repeat them so easy right but suppose you ask it instead

On Hacker News

Computerphile has an interesting video about Glitch Tokens, but as it goes with Computerphile videos it's also a good introduction to the topic. [VIDEO] — weinzierl, 2023-06
I found this tool recently when it was linked from this Computerphile video[1] about "glitch tokens". tldw: Certain junk data was thrown out post-tokenization e.g. the /r/counting[2] community data and debug logs from Rocket League some tokens specific to those contexts stuck around, however, and are now like "a color you've never seen before" as far as GPT-X models are concerned giving the model… — non-, 2023-04
That is indeed the funniest (and slightly horrifying) thing. Even if you understand "glitch tokens"[1] (which is BTW the best way I've seen to explain to nontechnicals (hi, Dad!!) and also some subset of technicals (hi various friends of mine!! +_+...) that LLM applied statistics models aren't anything like sentient AGI)... ...regardless, ChatGPT and its ilk nevertheless do often already outperfo… — veidr, 2023-06
It is well known, yes. This was discussed a way back during the whole glitch tokens deal: [VIDEO] A bunch of those, like " SolidGoldMagikarp" are Reddit users. — dale_glass, 2023-06
This reminds me of the recent so-called "Glitch Token" phenomenon[1]. When GPT-3 was presented with reserved tokens it never encountered during training, it reacted in extremely unpredictable ways -- often with a simple "fuck you". For those unfamiliar with LLM architecture: "tokens" are the smallest unit of lexical information available to the model. Common words often have their own token (e.g.… — chaorace, 2023-05

Related videos