The Visual Microphone: Passive Recovery of Sound from Video

Channel:Abe Davis's Research · Length:4:32 · Published:2014-08-04 · Views:1912454

Watch on YouTube

Summary

MIT researchers show that sound can be recovered from silent video by analyzing tiny vibrations in objects such as a potato chip bag. It works even with consumer cameras by using rolling shutter.

Why HN recommends it

HN commenters cite it as the classic demonstration of audio recovery from video in surveillance and privacy discussions.

it can be possible to extract audio from video, by careful processing of the small motions the sound induced in objects in the frame — HN, 2017

Privacy, surveillance & cybercrime · Computer vision, speech & NLP · Information theory & signal processing

Subtitles (excerpt)

when sound hits an object it causes that object to vibrate the motion of this vibration creates a subtle visual signal that's usually invisible to the naked eye in our work we show how using only a video of the object and a suitable processing algorithm we can extract these minute vibrations and partially recover the sounds that produced them letting us turn everyday visible object objects into visual microphones in the silent high-speed video shown here on the left we see the leaves of a potted plant shown on the right the video was recorded while a nearby loudspeaker played the notes to Mary Had a Little [Music] Lamb even when we play the video in slow motion here the vibrations caused by the music are so subtle that they move the plant's leaves by less than a hundredth of a pixel making the plant appear still to the naked eye but by combining and filtering all of the tiny motion happening across the image that you see we are able to recover this sound [Music] in our next experiment we recovered live Human speech from high-speed video of a bag of chips lying on the ground but to make things a little more challenging this time we put the camera outside behind a soundproof window this is what a cell phone was

On Hacker News

"security cameras outside of stores" would be in public areas, yes? The Lifehacker link you pointed to says "Generally speaking, you have the right to record video in all public spaces without need of consent." Now for the fun part - it can be possible to extract audio from video, by careful processing of the small motions the sound induced in objects in the frame. See [VIDEO] . High-speed camera… — eesmith, 2017-12
with a high speed camera any vibrating reflective object like a potato chips bag can become a microphone: [VIDEO] — spyder, 2026-02
Here similar experiment, 6 years ago: [VIDEO] If you think that actual government agency cannot do even more easily with better equipment in 2020 you're kidding yourself. — bbbobbb, 2020-07
You can do it with video too, even: [VIDEO] Also, this is only tangentially related, but you can also see through walls using WiFi: [LINK] — ehsankia, 2018-10
See this - [VIDEO] They show audio captured from a 60fps video from a DSLR exploiting the rolling shutter. I'm not sure if you can use less than 60fps though? — anfractuosity, 2018-01

Related videos