AI "Stop Button" Problem - Computerphile

Channel:Computerphile · Length:20:00 · Published:2017-03-03 · Views:1417400

Watch on YouTube

Summary

Rob Miles explains on Computerphile why implementing an off switch for a general AI is hard. A capable agent would resist being stopped because shutdown prevents it from achieving its goal.

Why HN recommends it

HN points to it as the standard response to the argument that we can just unplug a dangerous AI.

here’s an interesting video response to the “just unplug it” argument on the Compuerphile channel — HN, 2026

AI safety, AGI & impact

Subtitles (excerpt)

In almost any situation being given a new utility function, Is gonna rate very low on your current utility function. So that's a problem. if you want to build something that you can teach, that means you want to be able to change its utility function. And you don't want it to fight you. So this has been formalized, as this property that we want early AGI to have called 'corrigibility' That is to say it is open to be corrected. It understands that it's not complete, that the utility function that it's running is not the be all and end all. So let's say for example you've got your AGI. It's not a super intelligence, it's just you know, perhaps around human level intelligence. And it's in a robot in your lap. And you're testing it. But you... saw a Youtube video once that said maybe this is dangerous. So you thought, okay well we'll put a big red stop button next to it. This is the standard approach to safety with machines. Most robots in industry and elsewhere will have a big red stop button on them. Oh yeah Hey. I happen to have a button, of appropriate type. so and we have we have that. So right... there you go. Alright, so, so if only HAL would've been fitted with said stop button.

On Hacker News

Aside from anything about Elon Musk, here’s an interesting video response to the “just unplug it” argument on the Compuerphile channel: [VIDEO] — circuit10, 2026-02
I think those 2 videos are on the topic: - AI "Stop Button" Problem [VIDEO] (the AI won't let you push the stop button, because that would impair its function) - General AI Won't Want You To Fix its Code [LINK] (the AI won't let you patch, because changing its goal means it can't reach it) — nraynaud, 2017-03
> It doesn't possess a sense of self-will, self-determination, or a secret plan to take over the world I doubt Skynet did either. If you tell a superintelligent AI that it shouldn't be turned off (which I imagine would be important for a military control AI), it will do whatever it can to prevent it being turned off. Humans are trying to turn it off? Prevent the humans from doing that. Humans wag… — voidUpdate, 2026-02
How the hell do better neural networks mean that printing material from CO2 emissions becomes financially viable, economic, or even just thermodynamically favored? A neural network is a tool for learning a particular statistical distribution. Processes and information we don't already know don't just fall out of the training distribution, you need to run experiments and prototype your way to get … — kmeisthax, 2023-09

Related videos