Intelligence and Stupidity: The Orthogonality Thesis
Channel:Robert Miles AI Safety · Length:13:02 · Published:2018-01-11 · Views:711847
Summary
Robert Miles explains the orthogonality thesis: how an agent can be highly intelligent while pursuing goals that seem stupid or alien to humans. He separates intelligence from goals and touches on the stamp-collector example.
Why HN recommends it
HN people use it as the standard short explanation for why AI can understand human ethics without sharing them, and often recommend the whole channel.
this video to me states the problem in a nutshell (that whole channel is gold) — HN, 2023
Subtitles (excerpt)
Hi. This video is kind of a response to various comments that I've got over the years, ever since that video on Computerphile, where I was describing the sort of problems that we might have when we have a powerful artificial general intelligence with goals which aren't the same as our goals, even if those goals seem pretty benign. We used this thought experiment of an extremely powerful AGI working to optimize the simple goal of collecting stamps, and some of the problems that that might cause. I got some comments from people saying that they think the stamp collecting device is stupid, and not that it's a stupid thought experiment, but the device itself is actually stupid. They said unless it has complex goals or the ability to choose its own goals, then it didn't count as being highly intelligent. On other videos, I got comments saying it takes intelligence to do moral reasoning, so an intelligent AGI system should be able to do that, and a superintelligent should be able to do it better than humans. In fact, if a superintelligence decides that the right thing to do is to kill us all, then I guess that's the right thing to do. These comments are all kind of suffering from the same mistake, which is what this video is about.
On Hacker News
That’s the alignment problem. We don’t know what the actual goals of an AI trained neural net are. We know what criteria we trained it against, but it turns out that’s not at all the same thing. I highly recommend Rob Miles channel on YouTube. Here’s a good one, but they’re all fascinating. It turns out training an AI to have the actual goals we want it to have is fiendishly difficult. [VIDEO] — simonh, 2023-05
I've seen that paper, and it's amazing indeed what GPT-4 is capable of. But none of that supports that closing quote, which to me points to a worrying philosophical naïveté. How can one equip an entity with "intrinsic motivation"? By definition, if we have to equip it with it, that motivation is extrinsic. It belongs to the one who puts it there. A software engineer might decide to prompt his cre… — pegasus, 2023-03
Yes, I believe some careful and informed pondering leads one towards that conclusion. Hume's guillotine is an implacable philosophical device: [VIDEO] (I warmly recommend all the content on that delightfully frightening channel) — pegasus, 2023-03
> Isn’t just the ability to preform a task. Right. In this case I'd say it's the ability to interpret data and use it to succeed at whatever goals it has Evaluating emotional context would be similar to a chess engine calculating its next move. There's nothing there that implies a conscience, sentience, morals, feelings, suffering or anything 'human'. It's just a necessary intermediate function t… — AstroBen, 2025-04
> What about "most arbitrary goals are incompatible with human existence" requires super-human intelligence? The "taken to the absolute extreme" part. > A human who wanted to "build as many paperclips as possible" could cause a great deal of destruction today. Maybe, but a) no one really wants that (at least not as their only desire above all else) and b) we aren't superintelligent so it's hard t… — circuit10, 2023-08
Related videos
- Does Time Cause Gravity? · PBS Space Time · 11:51
- Classic Game Postmortem: Sid Meier's Civilization · GDC Festival of Gaming · 1:02:15
- A conversation with Elon Musk about Starship · Everyday Astronaut · 17:06
- Feynman :: Rules of Chess · defjam99b · 2:49
- Timothy Snyder: The Making of Modern Ukraine. Class 1: Ukrainian Questions Posed by Russian Invasion · YaleCourses · 46:05
- ChatGPT with Rob Miles - Computerphile · Computerphile · 36:01
- Explained: "Go Fast" UFO Video - Not Low and Not Fast - Like a Balloon! · Mick West · 6:42
- Tiedustelueverstin arvio Venäjästä | 3.12.2018 · fin_topsu · 1:00:22
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- 10 Reasons to Ignore AI Safety · Robert Miles AI Safety · 16:28
- Is AI Safety a Pascal's Mugging? · Robert Miles AI Safety · 13:41
- AI "Stop Button" Problem - Computerphile · Computerphile · 20:00
- MarI/O - Machine Learning for Video Games · SethBling · 5:57
- Deadly Truth of General AI? - Computerphile · Computerphile · 8:30
- The Gym of Life · Not Just Bikes · 11:47
- Intro to AI Safety, Remastered · Robert Miles AI Safety · 18:04