Nicholas Carlini - Black-hat LLMs | [un]prompted 2026

Channel:unprompted · Length:26:28 · Published:2026-03-25 · Views:390179

Watch on YouTube

Summary

Nicholas Carlini of Anthropic talks at [un]prompted 2026 about how adversaries could misuse current LLMs for attacks. He shows recent models finding zero-day vulnerabilities in heavily tested software, including a SQL injection in Ghost.

Why HN recommends it

HN cites it as a concrete demonstration that LLMs have suddenly become strong at finding and exploiting vulnerabilities.

This talk changed my view on where we're actually at — HN, 2026

Security & hacking · LLMs & generative AI

Subtitles (excerpt)

All right. So, before Nicholas speaks, I want to say a couple of words as well. I think that there are not that there are people in this industry who have been doing this for a long, long time truly understand this and can be considered the best. Now, who is the best one, the best two, the best three? I don't really know. But, we have the opportunity right now to listen to somebody who has not just been the best at this for the past year or the past two years, but for a long while and consistently makes the industry better just by doing his research, regardless of how much of that comes out. And I'll also tell you that he agreed last second and went through all the corporate uh PR nonsense that I don't know how much you have to do in Anthropic. It seems like you guys are easier than other people to manage to come here and give this talk to you today. So, I just want to get second for everybody for the last second, last minute getting into it thing and coming on stage and rushing in for handling truly in a global-level emergency or situation just to come and speak to us all. So, please, let's give it to Nicholas. Okay, thanks. Um yeah, so I'm Nicholas. Um I'm at Anthropic and I'm going to spend a little bit of time um just talking

On Hacker News

I'm sure they do, yet the models really are getting scarily good at this. This talk changed my view on where we're actually at: [VIDEO] — knowaveragejoe, 2026-04
This matches with what Nicholas Carlini from Anthropic said a the [un]prompted conference - [VIDEO] . Very worth watching. — wkandek, 2026-05
The talk "Black-Hat LLMs" just came out a few days ago: [VIDEO] Looks like LLMs are getting good at finding and exploiting these. — panstromek, 2026-04
This very question was asked to Nicholas Carlini from Anthropic at this talk: [VIDEO] The answer is complex, worth watching the video. But mainly, they don't know where to place the line. Defenders need tools, as good as attackers. Attackers will jailbreak models, defender might not, it's the safeguard positive in that case? Carlini actively asks the audience and community for "help" in determini… — ogig, 2026-04
[VIDEO] is the presentation itself. The prompts are trivial; the bug (and others) looks real and well-explained - I'm still skeptical but this looks a lot more real/useful than anything a year ago even suggested was possible... — eichin, 2026-04

Related videos