Connections between physics and deep learning
Channel:MITCBMM · Length:51:46 · Published:2016-08-30 · Views:66979
Summary
Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.
Why HN recommends it
HN recommends it as a clear argument for why deep and cheap learning succeeds, often alongside the associated paper.
physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits — HN, 2018
Machine learning & deep learning · Fundamental physics
Subtitles (excerpt)
MAX TEGMARK: So I'm going to talk about the connections between physics and deep learning. And I'm going to talk about this because I feel that there's a lot of really wonderful science to be done here in the middle of this Venn diagram, at the intersection between physics, neuroscience and AI. I'm going to give two different examples, in fact, of cool things exactly in that area. And I'm going to start with this question that Gabriel raised. Why is it that deep learning works so well? And moreover, why does cheap learning works well? I'll explain what I mean by that in a moment. So you've all heard a bunch of cool things about deep learning being successful here at the meeting. And Haim Sompolinsky also told you about why deep learning is sometimes better, and gave some very interesting partial answers. I'm going to argue that the full answer to this question cannot be found in mathematics and computer science alone, though. Because part of the answer is actually in physics. But before we go there, let's just remind ourselves, so we're all on the same page, what we mean by deep learning. So a feedforward neural network is just a function. A particular function, of course, where you send something in, in an input layer,
On Hacker News
Here's a related 2016 talk by Max Tegmark (second author) on connections between deep learning and physics: [VIDEO] The gist of it is that physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits. Random data does not have this property, and cannot be learned easily. Super fascinating. — tedsanders, 2018-05
I think the argument for using physics models for constraints in a DL system was a more clear argument [VIDEO] — jonmc12, 2017-01
You should watch Max Tegmark's talk "Connections between Physics and Deep Learning" if this interests you [1]. Additionally, a paper that has everyone excited about deep connections between the mathematical analysis of physical systems and the hierarchical feature learning paradigm speaks of the connection in terms of the Renormalization Group [2]. Regardless of it's practical utility, the philos… — adityab, 2016-09
See Tegmark's "why does deep cheap learning work so well" (well not so cheap anymore...) [VIDEO] — krackers, 2026-06
My random stream-of-consciousness reference reactions ... Tegmark & Lin's discussion of how deep learning maps onto the physical world: Why does deep and cheap learning work so well? [1][2] The work of Aerts et al. more than 10 years ago, including how vector space models of human categorization show QM structure [3]. Lucien Hardy's exquisite teasing out of the difference between classical and qu… — mikhailfranco, 2017-08
Related videos
- "Energy Revolution? More like a Crawl" - Dr. Vaclav Smil · Trottier Institute for Sustainability in Engineering and Design · 1:48:55
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- MarI/O - Machine Learning for Video Games · SethBling · 5:57
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- Computer program that learns to play classic NES games · suckerpinch · 16:17
- RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning · Google DeepMind · 1:28:12
- Let's build GPT: from scratch, in code, spelled out. · Andrej Karpathy · 1:56:20
- I bought 1000 meters of wire to settle a physics debate · AlphaPhoenix · 22:48
- Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning · Google TechTalks · 48:10
- A Hackers' Guide to Language Models · Jeremy Howard · 1:31:13
- The Speed of Light is NOT About Light · PBS Space Time · 12:46