"Turning the database inside out with Apache Samza" by Martin Kleppmann

Channel:Strange Loop Conference · Length:47:42 · Published:2014-09-21 · Views:206079

Watch on YouTube

Summary

Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.

Why HN recommends it

Commenters repeatedly cite it as the talk that reframed how they think about caching, derived data and event sourcing.

This talk on Apache Samza completely changed how I think about caching and derived data in general — HN, 2025

Databases & storage · Distributed systems

Subtitles (excerpt)

hi thank you so much for coming I'm hope you're having a really excellent strange Loop I'm I'm really enjoying it so far um I'm here to talk about turning the database inside out it's a talk about database architecture and application architecture it's somewhat related to an open source project that I've been working on over the last while called Apache samza um my name is Martin kman I was until recently at LinkedIn working on samza at the moment I'm taking a sabatical and actually writing a book for O'Reilly uh it's called designing data intensive applications it's got this awesome wild bore onr um the the idea of this book is to be a kind of a conceptual broad overview over how Data Systems actually work so looking at databases caches indexes batch processing stream processing how all of that fits together they're kind of big common ideas which which go through through all of those areas and if you can kind of understand those my thinking is we can all build better applications uh using though using those underlying ideas so um the early release we've just published the first few chapters of that as an early release just a few days ago um you can find it at data intensive. net in case you're interested anyway let's talk about databases and what I mean is

On Hacker News

Idk I think caching is a crucial part of many well-designed systems. There’s a lot of very cache-able data out there. If invalidating events are well defined or the data is fine being stale (week/month level dashboards, for example), that’s a fantastic reason to use a cache. I’d much rather just stuff those values in a cache than figure out any other more complicated solution. I also just think i… — chamomeal, 2025-08
Hey OP you may have seen this already, but in case you didn’t see my other comment, you should definitely check out this talk by Martin Kleppman. [VIDEO] It details Apache samza, which I didn’t totally grasp but it seems similar to what you’re talking about here. He talks about how if you could essentially use an event stream as your source of truth instead of a database, and you had a sufficient… — chamomeal, 2025-08
The talk on "Turning the database inside-out" [0][1] by the author, Martin Kleppmann, is a fantastic intro to these dynamics, and it's something I'll always recommend to both experienced and inexperienced data modelers and backend developers. It goes pedagogically through the way things are typically done in a relational database in such a clear way that word-for-word it's one of the best tutoria… — btown, 2019-05
Yes, I like to think of it as as the transaction log without any of the logic. If you’re interested, I can highly recommend Martin Kleppmann’s talk “Turning the database inside out” [VIDEO] — lbradstreet, 2018-05
Since this is taking off a bit: Here's an excellent talk that provides some context to the Confluent article, and takes us through the thought process of simplifying convoluted database-cache-user interactions by thinking in terms of streams of derived data: [VIDEO] . Also highly recommended. — btown, 2015-04

Related videos