Playing a Neural Network's version of GTA V: GAN Theft Auto
Channel:sentdex · Length:17:07 · Published:2021-06-18 · Views:766017
Summary
Sentdex demonstrates GAN Theft Auto, a GameGAN-based neural network that recreates GTA V so the network itself is the playable environment. He shows the results and releases the code.
Why HN recommends it
Cited as an early, striking demo of neural networks acting as game engines and world models.
This YouTube video, to me, shows the promise of things to come. — HN, 2022
Machine learning & deep learning · LLMs & generative AI · Game programming & engines
Subtitles (excerpt)
welcome everybody to gan theft auto what you see here is me playing inside of a neural networks environment this neural network was trained by watching a bot play grand theft auto 5 on this particular stretch of highway learning how to generate new frames from previous frames and player input so every pixel you see here is generated from a neural network while i play while generating the visuals or just improving them is interesting too and there has been some interesting work lately with neural networks improving the visuals with grand theft auto 5 in particular in attempts to make it look more photorealistic this is something much more than that the neural network in this case is the entire game it's handling what happens when you press a key to turn left or right there are no rules written here by us or the original grand theft auto engine the neural network is controlling all of this along with what happens when we try to turn into another car when we hit a wall or what objects in the background should do what the light reflections should do it's doing all of this based on our inputs it's the visuals the controls the physics it's everything we are playing entirely within a neural network's representation of grand
On Hacker News
What you're describing reminded me of this cool project: [VIDEO] "Playing a Neural Network's version of GTA V: GAN Theft Auto" — 7734128, 2024-08
> You could directly encode the elements as understanded by the process used to generate them, rather than attempting to reconstitute things from a stream of bitmaps. You might like Gan Theft Auto: [VIDEO] — eru, 2022-12
This YouTube video, to me, shows the promise of things to come. AI generated game worlds. Language models to generate plots and dialog, transformers and GANs to create illustrations. Imagine a game, a truly open world sand box, Grand Theft Auto meets AI Dungeon - every NPC is a "real" person with unlimited dialog options, the buildings you drive by you could easily walk in and investigate, unlimi… — ALittleLight, 2022-04
This video really amazed me in terms of automated game development. My fantasies are something like this and GPT-3 plus a couple decades of progress. I think we'll get to genuinely open ended sand box games. [VIDEO] — ALittleLight, 2021-08
>There's some progress in video generation which takes a short clip and extrapolates what happens next. That's a promising line of development. The key to "common sense" is being able to predict what happens next well enough to avoid big mistakes in the short term, a few seconds. How's that coming along? And what's the internal world model, assuming we even know? [VIDEO] This has been a thing for… — imtringued, 2025-11
Related videos
- The Antique Toaster that's Better than Yours · Technology Connections · 18:28
- Batteries aren't the only way to store energy. Here's another. · Technology Connections · 24:25
- The Epidemic of Fake Disease · Medlife Crisis · 20:57
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- Overwatch Gameplay Architecture and Netcode · GDC Festival of Gaming · 1:04:03
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- RustConf 2018 - Closing Keynote - Using Rust For Game Development by Catherine West · Rust Programming Language · 42:29
- A Deep Dive into Nanite Virtualized Geometry · SIGGRAPH Advances in Real-Time Rendering · 1:09:59
- How Crash Bandicoot Hacked The Original Playstation | War Stories | Ars Technica · Ars Technica · 31:37
- MarI/O - Machine Learning for Video Games · SethBling · 5:57
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- Jonathan Blow - How to program independent games - CSUA Speech · Alexandre Franco · 1:15:50