Building Software Systems At Google and Lessons Learned
Channel:Stanford · Length:1:22:45 · Published:2011-06-02 · Views:257383
Summary
Jeff Dean of Google describes his experiences building Google's systems and how its infrastructure evolved over time. The Stanford talk covers search, scaling tipping points and lessons from building large-scale software.
Why HN recommends it
HN recommends it as a classic look at systems engineering at Google, whose fundamentals have held up well despite its 2010 date.
It's from 2010, but the fundamentals probably haven't changed. — HN, 2020
Distributed systems · Tech company stories & documentaries
Subtitles (excerpt)
[Music] Stanford University. Okay. Can you hear me? Okay. Okay. Um, welcome to I guess this is E380, but it's also been sort of overridden with our distinguished lecture series. Um, today's speaker is Jeff Dean of Google. And um, you know, Jeff, you know, I don't want to use up all his time describing all his accomplishments, but I'll say he did get his degree at University of Washington, but since his adviser was a graduate of here, we can claim him as a grandchild of our institution, and we we certainly do that. Um he he initially landed after his degree at Deck um research lab worked there for a while and then made the smart move to jump over to Google in the early days where he was an instrumental in building a lot of the systems that Google uses to make money basically their search and language translation all all these things and along the way he was involved with some very influential in um pieces of infrastructure including big table map reduce and protocol buffers and so um that introduction, I'll let let Jeff describe them to you. Thank you, Malle. Um, so welcome. Uh, thank you for having me. Uh, I will try to speak louder. Um, so the plan for today is, uh, I'm going to connect my microphone.
On Hacker News
In addition to Linus's git talk, I really enjoyed Jeff Dean's EE380 retrospective on Building Systems at Google ( [VIDEO] ). Many people have mentioned Jeff's basic premise elsewhere ("Design a system for 10x your current need, but not 100x, rewrite it before then") but this talk gave several useful examples where tipping points occurred (at least with Search). — boulos, 2016-10
Probably this one: [LINK] Or this one: [VIDEO] (terrible audio) Both are very old, but extremely well aged. IMO, the secret behind Google Search is not the smartness of the algorithms, but how much of it is baked in those O(100ms) which takes Google machines to answer your query. That's why the links above are the true reason Google Search performs well. Natural Language Processing and Informatio… — mrazomor, 2022-08
There is a lecture from Jeff Dean about it: [VIDEO] It's from 2010, but the fundamentals probably haven't changed. — ma2rten, 2020-09
Also recommend Jeff Dean's tech talk Building Software Systems At Google and Lessons Learned which references those latency numbers. Slides: [LINK] Video: [VIDEO] Also, a previous thread on latency numbers: [LINK] — kqr2, 2012-12
Google is famous for publishing a lot of material every year. A couple of related publications you might be interested in: 1. The Datacenter as a computer: [LINK] 2. The SRE book: [LINK] A couple of dated presentations: 1. Jeff Dean on building software systems at Google (2010) at Stanford: [VIDEO] 2. Notes by James Hamilton on a talk by Jeff Dean on Google Datacenters in 2008: [LINK] -- Microsof… — ignoramous, 2019-02
Related videos
- Steve Jobs, "Computers are like a bicycle for our minds." - Michael Lawrence Films · Michael Lawrence · 1:39
- The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019 · GOTO Conferences · 42:02
- Micro Men - 720p (2009) · gimpymoo · 1:24:12
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- Project Code Rush - The Beginnings of Netscape / Mozilla Documentary · The Documentary Network · 56:08
- Ed Catmull, Pixar: Keep Your Crises Small · Stanford Graduate School of Business · 54:11
- Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp · Erlang Solutions · 49:15
- LISA11 - Fork Yeah! The Rise and Development of illumos · USENIX · 1:04:03
- "Turning the database inside out with Apache Samza" by Martin Kleppmann · Strange Loop Conference · 47:42
- The Language of the System - Rich Hickey · ClojureTV · 1:02:49
- Google I/O 2012 - Go Concurrency Patterns · Google for Developers · 51:27
- Conflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016 · GOTO Conferences · 51:35
- "Unison: a new distributed programming language" by Paul Chiusano · Strange Loop Conference · 41:05
- Paxos lecture (Raft user study) · Diego Ongaro · 1:06:33
- Secret History of Silicon Valley · Computer History Museum · 1:02:47
- Hewitt, Meijer and Szyperski: The Actor Model (everything you wanted to know...) · jasonofthel33t · 42:34