Movie reconstruction from human brain activity
Channel:Jack Gallant · Length:0:29 · Published:2011-09-22 · Views:6867281
Summary
Jack Gallant's lab demonstrates reconstructing movie trailer clips a subject watched from fMRI brain activity, using regression models trained on hours of viewing data. The video shows the original clip beside the reconstruction.
Why HN recommends it
HN cites it as the landmark brain-decoding demo and compares newer reconstruction work against it.
Reconstructing what a person is seeing out of their fMRI scan (2011) — HN, 2023
Computer vision, speech & NLP · Neuroscience & the brain
On Hacker News
This is a much cooler example [VIDEO] If I'm understanding the description correctly, they are just training it to recognize what the image is closest to and taking an slice of a youtube video that most closely matches it. I imagine if they used a more efficient method or trained it more, they could do way better. It seems like most of the data to build an accurate picture of what they are seeing… — Houshalter, 2013-08
I imagine the best case outcome would look something like Jack Gallant's 2011 work on visual mind reading, where they trained a model on brain activity watching hundreds of hours of YouTube and then attempted to reconstruct a view of video not in the training set by correlating the new brain activity with frames from the input... Sorry I'm not explaining very clearly but there's a YouTube video o… — jazzyjackson, 2024-12
I mean in 2011, they were already able to do some basic reconstruction: see [VIDEO] I could imagine improvements since then,especially with advances in image networks. — SubiculumCode, 2023-03
There was a video many years ago (early 2010s?) demoing a similar technology, which would overlay and blend many images on top of each other to make a fuzzy image approximating what was actually being viewed. Edit: found it! [VIDEO] — andai, 2023-03
Reminds me of this paper [1] from 2011. See it in action in [2] 1. [LINK] 2. [VIDEO] Edit: Just realized the paper above is also from Shinji Nishimoto — Lutzb, 2023-03
Related videos
- Sloane's Gap - Numberphile · Numberphile · 7:26
- The Immune System Explained I – Bacteria Infection · Kurzgesagt – In a Nutshell · 7:49
- DEF CON 25 - John Sotos - Genetic Diseases to Guide Digital Hacks of the Human Genome · DEFCONConference · 49:47
- Triangle of Power · 3Blue1Brown · 7:42
- Game Theory: How PewDiePie Conquered YouTube · The Game Theorists · 12:02
- 1. Introduction to Human Behavioral Biology · Stanford · 57:14
- Stanford's Sapolsky On Depression in U.S. (Full Lecture) · Stanford · 52:28
- What Bodies Think About: Bioelectric Computation Outside the Nervous System - NeurIPS 2018 · The Artificial Intelligence Channel · 52:06
- The great porn experiment | Gary Wilson | TEDxGlasgow · TEDx Talks · 16:29
- Controlling Your Dopamine For Motivation, Focus & Satisfaction · Andrew Huberman · 2:16:31
- Personal Growth Series: Dr. William Dement on Healthy Sleep and Optimal Perfo... · Google TechTalks · 59:57
- Joe Rogan Experience #1109 - Matthew Walker · PowerfulJRE · 1:55:32
- Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning · Google TechTalks · 48:10
- Master Your Sleep & Be More Alert When Awake · Andrew Huberman · 1:22:05
- What Alcohol Does to Your Body, Brain & Health · Andrew Huberman · 2:01:02
- "Voice Driven Development: Who needs a keyboard anyway?" by Emily Shea · Strange Loop Conference · 41:17