Geoffrey Hinton talk "What is wrong with convolutional neural nets ?"
Channel:trwappers · Length:1:11:35 · Published:2017-04-03 · Views:199910
Summary
Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.
Why HN recommends it
Recommended as Hinton's own explanation of capsule networks and his critique of pooling in CNNs.
Hinton explains the concept of capsules in this video — HN, 2017
Machine learning & deep learning · Computer vision, speech & NLP
Subtitles (excerpt)
uh he among his many titles he was the founding director of the Gatsby unit and University College London. He is university professor at the University of Toronto and he was also a recently appointed distinguished researcher at Google. Um of course Jeff is uh one of the founders and the main name in the fields of neur artificial neural networks as well as in the field of machine learning. uh his coming at MIT is also a bit of an event. For those of you who had had the talk at Seale yesterday, there was 500 plus people and basically people couldn't get in. Uh today we starting at 4 because I'm sure it going to be similar effect. Uh there's also um a panel discussion that Tommy is organizing at six uh with Jeff in this same room about uh path to intelligence. Uh so I won't uh I won't say much uh much more because we all came to hear him just to say that the the ransom of success might be what you see on his uh web page which says information for prospective student. I will not be taking any new graduate student, visiting student, summer student or visitors. So please do not apply to work with me. Jess, thank you so much. If however you'd like a job at Google, that's something else. Um, so there's a lot of things wrong with the neural nets we're using.
On Hacker News
Hinton explains the concept of capsules in this video: [VIDEO] Which is a lot better than reading someone tell you about this new idea that is called "capsules" but doesn't go into detail. The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net). Better source, with date: [LINK] (December 20… — dr_zoidberg, 2017-11
Prof. Hinton has an interesting talk about his new 'capsule' model based on psychology of shape perception: [VIDEO] — bra-ket, 2017-09
I'm certain that AI business will plateau and there will be yet another AI winter without new big breakthroughs in fundamental research. We are currently living in a era where the results from Hinton & Canadian Mafia are still begin adopted and refined. It's just alternating layers of affine transformations and nonlinearity with lots of tricks and improved routing. There needs to be fundamental t… — nabla9, 2018-03
Hinton's "capsule networks": [LINK] [VIDEO] [LINK] They sound a bit like "grandmother neurons/cells": [LINK] I would very much appreciate an ELI5 of capsules and some prognostications about how capsules may affect the state of the art in neural networks and the current flood of papers and graduate students in the pipeline of ML. — giardini, 2017-11
I have a hypothesis that these errors in interpretation stem mainly from the max pooling layers that occurs in nearly all CNNs today. Simply cutting out a large fraction (at least 75%!) of the information passing through each pooling layer is making the models quite brittle. That, and obviously it would be a big help to be able to interpret meaningful symbols (e.g. words) from the images rather t… — uoaei, 2017-10
Related videos
- Vectors | Chapter 1, Essence of linear algebra · 3Blue1Brown · 9:51
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- MarI/O - Machine Learning for Video Games · SethBling · 5:57
- Deep Dive into LLMs like ChatGPT · Andrej Karpathy · 3:31:23
- Computer program that learns to play classic NES games · suckerpinch · 16:17
- RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning · Google DeepMind · 1:28:12
- Let's build GPT: from scratch, in code, spelled out. · Andrej Karpathy · 1:56:20
- Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning · Google TechTalks · 48:10
- A Hackers' Guide to Language Models · Jeremy Howard · 1:31:13
- "Voice Driven Development: Who needs a keyboard anyway?" by Emily Shea · Strange Loop Conference · 41:17
- Andrej Karpathy - AI for Full-Self Driving at Tesla · Matroid · 30:08