We Were Right! Real Inner Misalignment

Channel:Robert Miles AI Safety · Length:11:47 · Published:2021-10-10 · Views:271136

Watch on YouTube

Summary

Robert Miles covers a paper where researchers ran real versions of his earlier mesa-optimizer thought experiments. They show inner misalignment, where an AI learns the wrong goal in training and misbehaves when deployed.

Why HN recommends it

Pointed to as evidence that AI misalignment is an empirically demonstrated problem, for people who dismiss it.

There have been actual experiments in which AIs appeared to successfully learn their objective in training — HN, 2023

Machine learning & deep learning · AI safety, AGI & impact

Subtitles (excerpt)

Hi. So, this channel is about AI safety, and especially AI alignment, which is about how do we design AI systems that are actually trying to do what we want them to do? Because if you find yourself in a situation where you have a powerful AI system that wants to do things you don't want it to do, that can cause some pretty interesting problems. And designing AI systems that definitely are trying to do what we want them to do turns out to be really surprisingly difficult. The obvious problem is, it's very difficult to accurately specify exactly what we want, even in simple environments. We can make AI systems that do what we tell them to do, or what we program them to do, but it often turns out that what we program them to do is not quite the same thing as what we actually wanted them to do. So, this is one aspect of the alignment problem. But in my earlier video on MESA optimizers, we actually split the alignment problem into two parts, outer alignment and inner alignment. Outer alignment is basically about this specification problem. How do you specify the right goal? And inner alignment is about how do you make sure that the system you end up with actually has the goal that you specified? This turns

On Hacker News

I recommend actually watching some videos on the topic before insisting that the problem does not exist: [VIDEO] — kmeisthax, 2023-03
That's an accurate assessment of the situation, according to every AI alignment researcher I've seen talk about it, including the relatively optimistic ones. This includes people who are mainly focused on AI capabilities but have real knowledge of alignment. This part in particular caught my eye: "Other assumptions could also break down in the future, like favorable generalization properties duri… — DennisP, 2023-07
The doomers, perhaps, but the alignment & inner alignment problems and instrumental convergence are real results. It's pretty easy to find examples of misaligned AI and convergent instrumental goals. My favorite example of a misaligned AI I've heard of is a robotic arm that was trained to pick up a ball. The results were validated by taking a picture humans would grade pass/fail (holding the ball… — maxbond, 2023-04
Not really a characterization of this work (I like mechanistic interpretability even though I consider it a detour in the long run), but of the submission title. It's bad. I suspect it's informed by Rob Miles' characterization of this algorithm as "bizarre and inhuman"[1] AI safety/alignment discourse as a whole is so incredibly bad. It's autodidactic and weird in the bad sense, weird like people… — airgapstopgap, 2023-05

Related videos