Conflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
Channel:GOTO Conferences · Length:51:35 · Published:2016-12-21 · Views:46323
Summary
Martin Kleppmann's GOTO Berlin 2016 talk on how collaborative editors, offline apps, and multi-datacenter databases handle concurrent updates. It covers conflict resolution approaches including operational transformation and CRDTs, with a worked look at text CRDTs.
Why HN recommends it
Treated as the go-to approachable yet deep primer on CRDTs and collaborative editing, recommended repeatedly in threads on Automerge, local-first software and OT versus CRDT.
Rest of the talk is also highly recommended for anyone interested in an approachable primer on CRDTs. — HN, 2019
Databases & storage · Distributed systems
Subtitles (excerpt)
(smooth electronic music) - Hello, everybody, thank you very much for coming. Good morning. Hope you are doing well. I'd like to talk today about conflict resolution in distributed systems, that is if several people change some data independently of each other, what happens, how do we resolve those conflicts that'll occur. My background is I'm a researcher at the University of Cambridge. I was previously in industry and a bunch of internet start ups. So I worked at Linkedin for a couple of years, for example. At the moment, I'm working on this research product called Trve Data. Spelled T-R-V-E and what we're trying to do here is to bring end to end encryption to a larger range of applications. So think something like Google Docs where several people can edit a document at the same time online but without having to trust Google servers because what we want to do is to be able to put data on various servers in the cloud but not have to worry about what happens if they get compromised or so on. So that's kind of the background of all of this. I'm not talking about the encryption and the security protocols today, I'm only focusing on one little piece of that whole project which is what happens if several people edit data
On Hacker News
> Indeed, the literature of CRDT does specify a mathematically correct answer. But this does not always line up with what humans would find the most faithful rendering of intent. This is a very salient point that anyone thinking of using CRDTs to "solve" synchronization in an user-facing application needs to take into consideration. Yes, CRDTs will guarantee that clients converge to an identical,… — lewisl9029, 2019-05
CRDTs are quite actively discussed in the IPFS community, in particular here: [LINK] (lots of great resources if you're interested in CRDTs) Also check out automerge, a CRDT that aims to be as JSON-like as possible: [LINK] Martin Kleppmann (one of the creators of Automerge) has given several great in-depth talks about it, eg. [VIDEO] — lachenmayer, 2019-03
CRDTs are great, and I've seen a lot of quality in Christopher's work. A few other resources for those interested in CRDT subject: - Our cartoon explainer [LINK] - An excellent talk by Martin Kleppmann [VIDEO] - Marc Shapiro's in depth talk [LINK] — marknadal, 2019-03
I've spent almost a decade working on CRDTs, largely from an industry perspective but also some academic side. They're great, and not too hard to understand if you take time to think about them. Here are some animated/interactive cartoon explainers we've made to help teach the concepts: - How a CRDT version of Operational Transformation (Google Docs) works: [LINK] - Why CRDTs are better than cent… — marknadal, 2018-02
Is this website related to the popular [LINK] ? Holy cow, this is an intense article, with great definitions for a bunch of detailed variations of distribute system jargon, and even has some nice images to explain the ideas! My specialty is in eventually consistent data types, which there section is much smaller on, but they have some links to follow up on CRDT stuff. > are often limited in funct… — marknadal, 2017-09
Related videos
- React Helsinki February 2020 · Fraktio · 1:16:07
- In Search of Tomorrow · Berkeley Institute for Data Science (BIDS) · 56:26
- Bottom Up vs Top Down Design in Clojure - Mark Bastian · ClojureTV · 36:42
- Tree-sitter: a new parsing system for programming tools - GitHub Universe 2017 · GitHub · 43:00
- The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019 · GOTO Conferences · 42:02
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- The Value of Values with Rich Hickey · InfoQ · 31:43
- AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) · Amazon Web Services · 59:56
- Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp · Erlang Solutions · 49:15
- "Turning the database inside out with Apache Samza" by Martin Kleppmann · Strange Loop Conference · 47:42
- The Language of the System - Rich Hickey · ClojureTV · 1:02:49
- Google I/O 2012 - Go Concurrency Patterns · Google for Developers · 51:27
- "Unison: a new distributed programming language" by Paul Chiusano · Strange Loop Conference · 41:05
- 01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019) · CMU Database Group · 1:06:44
- Paxos lecture (Raft user study) · Diego Ongaro · 1:06:33
- Introduction to NoSQL • Martin Fowler • GOTO 2012 · GOTO Conferences · 54:51