"Turning the database inside out with Apache Samza" by Martin Kleppmann
Channel:Strange Loop Conference · Length:47:42 · Published:2014-09-21 · Views:206079
Summary
Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.
Why HN recommends it
Commenters repeatedly cite it as the talk that reframed how they think about caching, derived data and event sourcing.
This talk on Apache Samza completely changed how I think about caching and derived data in general — HN, 2025
Databases & storage · Distributed systems
Subtitles (excerpt)
hi thank you so much for coming I'm hope you're having a really excellent strange Loop I'm I'm really enjoying it so far um I'm here to talk about turning the database inside out it's a talk about database architecture and application architecture it's somewhat related to an open source project that I've been working on over the last while called Apache samza um my name is Martin kman I was until recently at LinkedIn working on samza at the moment I'm taking a sabatical and actually writing a book for O'Reilly uh it's called designing data intensive applications it's got this awesome wild bore onr um the the idea of this book is to be a kind of a conceptual broad overview over how Data Systems actually work so looking at databases caches indexes batch processing stream processing how all of that fits together they're kind of big common ideas which which go through through all of those areas and if you can kind of understand those my thinking is we can all build better applications uh using though using those underlying ideas so um the early release we've just published the first few chapters of that as an early release just a few days ago um you can find it at data intensive. net in case you're interested anyway let's talk about databases and what I mean is
On Hacker News
Idk I think caching is a crucial part of many well-designed systems. There’s a lot of very cache-able data out there. If invalidating events are well defined or the data is fine being stale (week/month level dashboards, for example), that’s a fantastic reason to use a cache. I’d much rather just stuff those values in a cache than figure out any other more complicated solution. I also just think i… — chamomeal, 2025-08
Hey OP you may have seen this already, but in case you didn’t see my other comment, you should definitely check out this talk by Martin Kleppman. [VIDEO] It details Apache samza, which I didn’t totally grasp but it seems similar to what you’re talking about here. He talks about how if you could essentially use an event stream as your source of truth instead of a database, and you had a sufficient… — chamomeal, 2025-08
The talk on "Turning the database inside-out" [0][1] by the author, Martin Kleppmann, is a fantastic intro to these dynamics, and it's something I'll always recommend to both experienced and inexperienced data modelers and backend developers. It goes pedagogically through the way things are typically done in a relational database in such a clear way that word-for-word it's one of the best tutoria… — btown, 2019-05
Yes, I like to think of it as as the transaction log without any of the logic. If you’re interested, I can highly recommend Martin Kleppmann’s talk “Turning the database inside out” [VIDEO] — lbradstreet, 2018-05
Since this is taking off a bit: Here's an excellent talk that provides some context to the Confluent article, and takes us through the thought process of simplifying convoluted database-cache-user interactions by thinking in terms of streams of derived data: [VIDEO] . Also highly recommended. — btown, 2015-04
Related videos
- Distributed Systems 1.1: Introduction · Martin Kleppmann · 14:35
- Lecture 1: Introduction · MIT 6.824: Distributed Systems · 1:19:35
- 01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019) · CMU Database Group · 1:06:44
- Preface | MIT Calculus Revisited: Single Variable Calculus · MIT OpenCourseWare · 32:06
- The Next Major RTS Will Fail. This Is Why. · GiantGrantGames · 25:09
- How We Made MuseScore 4 - Music App Design is Challenging! · Tantacrul · 46:33
- CheerpX: a WebAssembly-based x86 virtual machine in the browser, Yuri Iozzelli · WebAssembly NA · 37:10
- NaN Gates and Flip FLOPS · suckerpinch · 19:07
- The Closest Thing We Have to Alien Technology · Veritasium · 55:00
- This Invention Made Disney MILLIONS, but Then They LOST It! · Corridor Crew · 12:25
- Building The World's First Computer - Greeks, Clocks and Rockets - Antikythera Mechanism Episode 1 · Clickspring · 6:52
- EPC2018 | Wave Function Collapse in Bad North | Oskar Stalberg · BUas Games · 37:41
- Music Software & Bad Interface Design: Avid’s Sibelius · Tantacrul · 21:35
- "Simple Made Easy" - Rich Hickey (2011) · Strange Loop Conference · 1:01:38
- "Cursorless: A spoken language for editing code" by Pokey Rule (Strange Loop 2023) · Strange Loop Conference · 39:26
- Floating Point Numbers - Computerphile · Computerphile · 9:16
- Net Neutrality: Last Week Tonight with John Oliver (HBO) · LastWeekTonight · 13:18
- De-fragmenting C++: Making Exceptions and RTTI More Affordable and Usable - Herb Sutter CppCon 2019 · CppCon · 1:33:12