Nicholas Carlini - Black-hat LLMs | [un]prompted 2026
Channel:unprompted · Length:26:28 · Published:2026-03-25 · Views:390179
Summary
Nicholas Carlini of Anthropic talks at [un]prompted 2026 about how adversaries could misuse current LLMs for attacks. He shows recent models finding zero-day vulnerabilities in heavily tested software, including a SQL injection in Ghost.
Why HN recommends it
HN cites it as a concrete demonstration that LLMs have suddenly become strong at finding and exploiting vulnerabilities.
This talk changed my view on where we're actually at — HN, 2026
Security & hacking · LLMs & generative AI
Subtitles (excerpt)
All right. So, before Nicholas speaks, I want to say a couple of words as well. I think that there are not that there are people in this industry who have been doing this for a long, long time truly understand this and can be considered the best. Now, who is the best one, the best two, the best three? I don't really know. But, we have the opportunity right now to listen to somebody who has not just been the best at this for the past year or the past two years, but for a long while and consistently makes the industry better just by doing his research, regardless of how much of that comes out. And I'll also tell you that he agreed last second and went through all the corporate uh PR nonsense that I don't know how much you have to do in Anthropic. It seems like you guys are easier than other people to manage to come here and give this talk to you today. So, I just want to get second for everybody for the last second, last minute getting into it thing and coming on stage and rushing in for handling truly in a global-level emergency or situation just to come and speak to us all. So, please, let's give it to Nicholas. Okay, thanks. Um yeah, so I'm Nicholas. Um I'm at Anthropic and I'm going to spend a little bit of time um just talking
On Hacker News
I'm sure they do, yet the models really are getting scarily good at this. This talk changed my view on where we're actually at: [VIDEO] — knowaveragejoe, 2026-04
This matches with what Nicholas Carlini from Anthropic said a the [un]prompted conference - [VIDEO] . Very worth watching. — wkandek, 2026-05
The talk "Black-Hat LLMs" just came out a few days ago: [VIDEO] Looks like LLMs are getting good at finding and exploiting these. — panstromek, 2026-04
This very question was asked to Nicholas Carlini from Anthropic at this talk: [VIDEO] The answer is complex, worth watching the video. But mainly, they don't know where to place the line. Defenders need tools, as good as attackers. Attackers will jailbreak models, defender might not, it's the safeguard positive in that case? Carlini actively asks the audience and community for "help" in determini… — ogig, 2026-04
[VIDEO] is the presentation itself. The prompts are trivial; the bug (and others) looks real and well-explained - I'm still skeptical but this looks a lot more real/useful than anything a year ago even suggested was possible... — eichin, 2026-04
Related videos
- Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller Explains · Perimeter Institute for Theoretical Physics · 1:15:10
- Behind the Scenes of iOS Security · Black Hat · 51:31
- Why Electronic Voting Is Still A Bad Idea · Tom Scott · 12:00
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Defcon 18 Changing threats to privacy Moxie Marlinspike Part · killab66661 · 42:50
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- 28c3: The Science of Insecurity · 28c3 · 59:23
- Let's build GPT: from scratch, in code, spelled out. · Andrej Karpathy · 1:56:20
- DEF CON 25 - John Sotos - Genetic Diseases to Guide Digital Hacks of the Human Genome · DEFCONConference · 49:47
- Elevator Hacking: From the Pit to the Penthouse · DeviantOllam · 1:55:53
- A Hackers' Guide to Language Models · Jeremy Howard · 1:31:13
- Compiling C to printable x86, to make an executable research paper · suckerpinch · 25:41
- Break Me00 The MoVfuscator Turning mov into a soul crushing RE nightmare Christopher Domas · Adrian Crenshaw · 46:19
- BlackHat USA 2011: SSL And The Future Of Authenticity · Christiaan008 · 48:01
- 27C3: Adventures in analyzing Stuxnet (Bruce Dang from Microsoft) · basti0007 · 1:14:56