The Visual Microphone: Passive Recovery of Sound from Video
Channel:Abe Davis's Research · Length:4:32 · Published:2014-08-04 · Views:1912454
Summary
MIT researchers show that sound can be recovered from silent video by analyzing tiny vibrations in objects such as a potato chip bag. It works even with consumer cameras by using rolling shutter.
Why HN recommends it
HN commenters cite it as the classic demonstration of audio recovery from video in surveillance and privacy discussions.
it can be possible to extract audio from video, by careful processing of the small motions the sound induced in objects in the frame — HN, 2017
Privacy, surveillance & cybercrime · Computer vision, speech & NLP · Information theory & signal processing
Subtitles (excerpt)
when sound hits an object it causes that object to vibrate the motion of this vibration creates a subtle visual signal that's usually invisible to the naked eye in our work we show how using only a video of the object and a suitable processing algorithm we can extract these minute vibrations and partially recover the sounds that produced them letting us turn everyday visible object objects into visual microphones in the silent high-speed video shown here on the left we see the leaves of a potted plant shown on the right the video was recorded while a nearby loudspeaker played the notes to Mary Had a Little [Music] Lamb even when we play the video in slow motion here the vibrations caused by the music are so subtle that they move the plant's leaves by less than a hundredth of a pixel making the plant appear still to the naked eye but by combining and filtering all of the tiny motion happening across the image that you see we are able to recover this sound [Music] in our next experiment we recovered live Human speech from high-speed video of a bag of chips lying on the ground but to make things a little more challenging this time we put the camera outside behind a soundproof window this is what a cell phone was
On Hacker News
"security cameras outside of stores" would be in public areas, yes? The Lifehacker link you pointed to says "Generally speaking, you have the right to record video in all public spaces without need of consent." Now for the fun part - it can be possible to extract audio from video, by careful processing of the small motions the sound induced in objects in the frame. See [VIDEO] . High-speed camera… — eesmith, 2017-12
with a high speed camera any vibrating reflective object like a potato chips bag can become a microphone: [VIDEO] — spyder, 2026-02
Here similar experiment, 6 years ago: [VIDEO] If you think that actual government agency cannot do even more easily with better equipment in 2020 you're kidding yourself. — bbbobbb, 2020-07
You can do it with video too, even: [VIDEO] Also, this is only tangentially related, but you can also see through walls using WiFi: [LINK] — ehsankia, 2018-10
See this - [VIDEO] They show audio captured from a 60fps video from a DSLR exploiting the rolling shutter. I'm not sure if you can use less than 60fps though? — anfractuosity, 2018-01
Related videos
- I Watched Ancient Apocalypse So You Don't Have To (Part 1) · Miniminuteman · 54:26
- The Secret Life of Machines - Vacuum Cleaners 1/2 · Buddy Runyan · 13:04
- Why Canadians Can't Bike in the Winter (but Finnish people can) · Not Just Bikes · 15:13
- But what is the Fourier Transform? A visual introduction. · 3Blue1Brown · 19:41
- D/A and A/D | Digital Show and Tell (Monty Montgomery @ xiph.org) · FL Studio · 23:52
- Defcon 18 Changing threats to privacy Moxie Marlinspike Part · killab66661 · 42:50
- The Fast Fourier Transform (FFT): Most Ingenious Algorithm Ever? · Reducible · 28:23
- This Video Is Sponsored By ███ VPN · Tom Scott · 7:25
- America Keeps Rejecting an ID Card It Already Has · CGP Grey · 7:15
- Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning · Google TechTalks · 48:10
- Frank Abagnale | Catch Me If You Can | Talks at Google · Talks at Google · 1:03:19
- #HITB2012KUL D1T3 - The Grugq - OPSEC: Because Jail is for wuftpd · Hack In The Box Security Conference · 1:04:25
- Cybersecurity as Realpolitik by Dan Geer presented at Black Hat USA 2014 · Black Hat · 54:35
- "Voice Driven Development: Who needs a keyboard anyway?" by Emily Shea · Strange Loop Conference · 41:17
- Andrej Karpathy - AI for Full-Self Driving at Tesla · Matroid · 30:08
- 28c3: The coming war on general computation · 28c3 · 54:34