Connections between physics and deep learning

Channel:MITCBMM · Length:51:46 · Published:2016-08-30 · Views:66979

Watch on YouTube

Summary

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

Why HN recommends it

HN recommends it as a clear argument for why deep and cheap learning succeeds, often alongside the associated paper.

physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits — HN, 2018

Machine learning & deep learning · Fundamental physics

Subtitles (excerpt)

MAX TEGMARK: So I'm going to talk about the connections between physics and deep learning. And I'm going to talk about this because I feel that there's a lot of really wonderful science to be done here in the middle of this Venn diagram, at the intersection between physics, neuroscience and AI. I'm going to give two different examples, in fact, of cool things exactly in that area. And I'm going to start with this question that Gabriel raised. Why is it that deep learning works so well? And moreover, why does cheap learning works well? I'll explain what I mean by that in a moment. So you've all heard a bunch of cool things about deep learning being successful here at the meeting. And Haim Sompolinsky also told you about why deep learning is sometimes better, and gave some very interesting partial answers. I'm going to argue that the full answer to this question cannot be found in mathematics and computer science alone, though. Because part of the answer is actually in physics. But before we go there, let's just remind ourselves, so we're all on the same page, what we mean by deep learning. So a feedforward neural network is just a function. A particular function, of course, where you send something in, in an input layer,

On Hacker News

Here's a related 2016 talk by Max Tegmark (second author) on connections between deep learning and physics: [VIDEO] The gist of it is that physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits. Random data does not have this property, and cannot be learned easily. Super fascinating. — tedsanders, 2018-05
I think the argument for using physics models for constraints in a DL system was a more clear argument [VIDEO] — jonmc12, 2017-01
You should watch Max Tegmark's talk "Connections between Physics and Deep Learning" if this interests you [1]. Additionally, a paper that has everyone excited about deep connections between the mathematical analysis of physical systems and the hierarchical feature learning paradigm speaks of the connection in terms of the Renormalization Group [2]. Regardless of it's practical utility, the philos… — adityab, 2016-09
See Tegmark's "why does deep cheap learning work so well" (well not so cheap anymore...) [VIDEO] — krackers, 2026-06
My random stream-of-consciousness reference reactions ... Tegmark & Lin's discussion of how deep learning maps onto the physical world: Why does deep and cheap learning work so well? [1][2] The work of Aerts et al. more than 10 years ago, including how vector space models of human categorization show QM structure [3]. Lucien Hardy's exquisite teasing out of the difference between classical and qu… — mikhailfranco, 2017-08

Related videos