Building the Software 2 0 Stack (Andrej Karpathy)
Channel:Databricks · Length:17:54 · Published:2018-08-29 · Views:72933
Summary
Andrej Karpathy's Databricks talk on Software 2.0, the shift from human-written code to code produced by neural network training. It covers how developer work moves toward curating datasets, the advantages and challenges, and opportunities for new tooling.
Why HN recommends it
HN uses it as the primary source for the 'Software 2.0' idea and recommends it as a quick talk on the next software stack.
here's a good/quick talk by Andrej Kapathy on what it will take to build the next software stack. — HN, 2020
Machine learning & deep learning · Tech pioneers & interviews
Subtitles (excerpt)
great okay so I'm excited to be here I'd like to tell you a little bit about what I see as a transformation in how we write software so as referring to software 2.0 actually to tell the story correctly I actually have to go back quite a bit of time about 1 million years back this is us throwing rocks at each other 1 million years ago but we're actually pretty clever species we discovered how to control fire and developed our agriculture and then we started building machines we use those machines for computation and we transitioned the computation from mechanical to electrical and then here we are 2018 and we're writing software and we've built a pretty complex software tools and the way we've come about all of this progress is that we actually apply a single kind of approach to - to advancing all of these applications so first we identified some kind of a problem and then we break it down to subproblems and then we create we create algorithms for every one of those subproblems and then we stack them together up into a system and then those systems can actually get pretty elaborate and pretty complex and so this has this really kind of approach by decomposition has worked for us very well and we've been able to
On Hacker News
Btw, for anyone interested, here's a good/quick talk by Andrej Kapathy on what it will take to build the next software stack. [VIDEO] — __sy__, 2020-07
Was there a dedicated HN submission for this topic? This is extremely fascinating to me. In what Andrej Karpathy calls "software 2.0"[1], leaking your model is equivalent to leaking the main IP of your company. Unlike source code where it loses value quickly out of context (e.g. twitch leaked their code yet that didn't spawn a bunch of twitch clones), models can be fine-tuned and transfer-learnin… — gfd, 2022-10
Andrej Karpathy (who was one of the presenters today) actually made similar statements last year, about how the part of the software stack that is not neural networks is gradually shrinking: [VIDEO] — wcoenen, 2019-04
Related videos
- How does Starlink Satellite Internet Work?📡☄🖥 · Branch Education · 28:08
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- But what is a neural network? | Deep learning chapter 1 · 3Blue1Brown · 18:40
- Jim Simons (full length interview) - Numberphile · Numberphile2 · 1:00:43
- Steve Jobs Stanford Commencement Speech 2005 · JoshuaG · 14:34
- AlphaGo - The Movie | Full award-winning documentary · Google DeepMind · 1:30:28
- Transformers, the tech behind LLMs | Deep Learning Chapter 5 · 3Blue1Brown · 27:14
- Alan Kay - Normal Considered Harmful · powerfuloutlet · 1:07:26
- Growing a Language, by Guy Steele · Bill Pugh · 53:31
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · USENIX · 51:22
- A Philosophy of Software Design | John Ousterhout | Talks at Google · Talks at Google · 1:01:40
- "Uncle" Bob Martin - "The Future of Programming" · Paul Stringer's Mobile Tech · 1:18:21
- VCF East: Ken Thompson interviewed by Brian Kernighan · Vintage Computer Federation · 1:03:51
- Lecture 1 | Machine Learning (Stanford) · Stanford · 1:08:40
- Hamming, "You and Your Research" (June 6, 1995) · securitylectures · 44:02
- MarI/O - Machine Learning for Video Games · SethBling · 5:57