How We've Scaled Dropbox
Channel:Stanford · Length:1:08:16 · Published:2012-09-10 · Views:372798
Summary
Dropbox engineer Kevin Modzelewski gives a Stanford colloquium on Dropbox's history and the technical problems it faced scaling, including detailed database schema decisions.
Why HN recommends it
Recommended as a detailed, candid case study of scaling a real system from zero users to millions.
I found the Dropbox lecture [1] at Stanford one of the most riveting things ever. — HN, 2018
Databases & storage · Distributed systems · Cloud, DevOps & reliability
Subtitles (excerpt)
[Music] Stanford University Welcome to dou 380 winter 201122 I'm Andy Freeman the other course organizer is Dennis Allison we're approaching the end of the quarter so if you're taking the class for credit please caught up remember no incompletes um we've talked a lot about large systems and scalable systems but we haven't talked much if at all about rapidly growing systems and rapid growth is the goal of most startups but it can be extremely hard both financially and technically today's speaker Ted meleski of Dropbox had the Good Fortune to have extremely rapid demand growth and today's talk is about how Dropbox dealt with technical challenges with very minimal resources thanks so hi everyone uh my name is Kevin majeski and um I'm the server team lead at Dropbox uh server team is a little bit of a historical name uh as I'll talk about a little bit later uh but we're responsible for the architecture and evolution of the Dropbox back end which is what I'm here to talk to you guys today about uh so the rough structure of the talk is first an introduction about what this talk is then uh some background about Dropbox what dropboxes what kinds of technical challenges we Face uh which will give some insight
On Hacker News
I found the Dropbox lecture [1] at Stanford one of the most riveting things ever. There is just so much technology behind Dropbox, it is staggering. There is a reason why it is so much better than iCloud sync, Google Drive, Box or OneDrive. [1] [VIDEO] — fermienrico, 2018-09
imo the best way is to look at what other's have built. Here's some of my favorite talks that go from 0 users to millions. Dropbox - [VIDEO] Instagram - [LINK] Slack (bonus. not as applicable but good reminder of why initial architecture does matter) - [LINK] — testb, 2018-06
Dropbox is a lot more than a GUI on top of rsync. Even purely from an engineering standpoint (ignoring product & design) that's incorrect. You might enjoy this talk: [VIDEO] — jesseendahl, 2018-03
Applications dont really need to be well architected until they are hitting scale. Then the parts of their system that need to relieve pressure will need to be re-architected. This is almost like a case study and there are a lot of good talks on youtube from places like dropbox and facebook that explain the problem and solution. Example: [VIDEO] If you dont want to do youtube case studies there a… — bradgnar, 2017-07
This is a talk about the evolution of Dropbox's architecture from 2012: [VIDEO] . It is incredibly detailed (down to the exact sql schema they use for file metadata etc). — rawnlq, 2014-10
Related videos
- How China Tracks Everyone · VICE News · 12:55
- Behind the Scenes of iOS Security · Black Hat · 51:31
- Steve Jobs Stanford Commencement Speech 2005 · JoshuaG · 14:34
- The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019 · GOTO Conferences · 42:02
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- The Value of Values with Rich Hickey · InfoQ · 31:43
- Velocity NY 2013: Richard Cook, "Resilience In Complex Adaptive Systems" · O'Reilly · 19:09
- Debugging Under Fire: Keep your Head when Systems have Lost their Mind • Bryan Cantrill • GOTO 2017 · GOTO Conferences · 52:34
- AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) · Amazon Web Services · 59:56
- Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp · Erlang Solutions · 49:15
- "Turning the database inside out with Apache Samza" by Martin Kleppmann · Strange Loop Conference · 47:42
- The Language of the System - Rich Hickey · ClojureTV · 1:02:49
- Google I/O 2012 - Go Concurrency Patterns · Google for Developers · 51:27
- Literate Devops with Emacs · Howard Abrams · 28:36
- Conflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016 · GOTO Conferences · 51:35
- "Unison: a new distributed programming language" by Paul Chiusano · Strange Loop Conference · 41:05