Why Google Stores Billions of Lines of Code in a Single Repository
Channel:@Scale · Length:30:50 · Published:2015-09-14 · Views:317620
Summary
Rachel Potvin describes the scale of Google's codebase and its custom monolithic source repository. She explains the reasons for the monorepo model, the tooling that makes it workable, and its tradeoffs.
Why HN recommends it
The go-to reference in monorepo and trunk-based development discussions, recommended as a firsthand account of how Google manages source control.
I'd recommend this video that describes it. — HN, 2017
Software architecture & design · Testing, debugging & developer tools
Subtitles (excerpt)
join me in welcoming Rachel from Google thanks a lot Brian my name is Rachel puffin and I'm an engineering manager at Google and as some of the more observant amongst you might have noticed I'm also extremely pregnant so I'm expecting a baby next week so that's why the chairs here just in case oh okay misinterpreted um so today I'm going to talk to you about Google's strategy for source control management and probably many of you have heard that we use one giant single shared code base at Google and probably at first glance some of you think that this is actually insane so the goal of this talk is to actually explain to you the reasons behind why we've chosen this approach so I started my career in the video game industry where I worked as a software developer for many years and back then the company I worked on would be working on multiple games at the same time and each game would be in his own separate repository and lots of times these games would be built on top of the same game engines so we would have one copy of the game engine code in each of our repositories and these would sort of evolve independently and then without fail some manager somewhere would decide that one feature that had been
On Hacker News
Googler opinions are my own. I also highly recommend this ACM article talking about our source control from 2016. [LINK] There's also a YouTube video which is linked from the wired article that is good. [VIDEO] — kyrra, 2022-05
Google doesn't use perforce anymore. They have a custom in-house SCM that has various interfaces wrapped around it. I'd recommend this video that describes it. [VIDEO] — kyrra, 2017-01
It's common practice for folks building Go binaries (but not libraries) to check in the code for their dependencies. So whenever you build Docker, you get the same thing unless Docker made a change (either by changing Docker itself or explicitly checking in an upgrade to a dep). There are a few nice things about it; vendoring means you won't have a dependency on other repos or archives being avai… — twotwotwo, 2016-07
This paper from last year surveys Google “engineers who have experience with both monolithic repos and multiple, per-project repos” to see what exactly they like and dislike about monorepos. The results match my experience, and I find the paper well-written (especially like the “Threats to Validity” section that says what biases may have affected the results; more papers and arguments should have… — svat, 2019-03
What I don't understand is how they accomplish larger collaborative changes. The paper says: "Almost all development occurs at the 'head' of the repository, not on branches." Googler Rachel Potvin made an even stronger statement in her presentation about "The Motivation for a Monolithic Codebase" [1]: "Branching for development at Google is exceedingly rare [..]" In the related ACM paper she publ… — weinzierl, 2019-01
Related videos
- Google Self-Driving Car Project | SXSW Interactive 2016 · SXSW · 52:21
- Nest Protect is a terrible buggy product · Brad Fitzpatrick · 5:57
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47
- "Simple Made Easy" - Rich Hickey (2011) · Strange Loop Conference · 1:01:38
- "Making Impossible States Impossible" by Richard Feldman · elm-conf · 25:06
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- 🚀 TDD, Where Did It All Go Wrong (Ian Cooper) · DevTernity Conference · 1:03:55
- Bret Victor - Inventing on Principle · Rui Oliveira · 54:20
- How To Design A Good API and Why it Matters · Google TechTalks · 1:00:19
- The Value of Values with Rich Hickey · InfoQ · 31:43
- Overwatch Gameplay Architecture and Netcode · GDC Festival of Gaming · 1:04:03
- Solving Problems the Clojure Way - Rafal Dittwald · Clojure/north · 1:02:26
- Lecture 1A: Overview and Introduction to Lisp · MIT OpenCourseWare · 1:12:55
- Debugging Under Fire: Keep your Head when Systems have Lost their Mind • Bryan Cantrill • GOTO 2017 · GOTO Conferences · 52:34
- "Performance Matters" by Emery Berger · Strange Loop Conference · 42:15
- Alan Kay - Normal Considered Harmful · powerfuloutlet · 1:07:26