Scaling the Linear Sync Engine
Channel:Linear · Length:1:06:07 · Published:2023-06-29 · Views:37224
Summary
A Linear engineer talks about scaling the company's sync engine, which keeps client data local and synchronized. It follows an earlier talk on the same system.
Why HN recommends it
Recommended whenever local-first software, sync engines and CRDT alternatives come up, as a practical production account.
This talk by Karri about Linear's "sync engine" is also a good watch — HN, 2023
Web development · Software architecture & design · Distributed systems
Subtitles (excerpt)
Welcome to my second real-time sync talk. If you haven't seen uh the first one, who has seen the first one that I've done? Almost everybody. Awesome. I'll do a quick recap on on on that talk cuz it sort of talks about the internals of how the sync engine works. Um and this talk sort of is a continuation of that. Um like I want to sort of go over sort of the challenges that we've had scaling uh the sync engine and um how it actually works um in in you know in the Linear application and what kind of changes we've made and maybe you know how the API around it has has formed you know because of these you know scaling challenges. Um and um you know to start things off with um we always want to build on strong foundations. Um the that you know holds true for the sync engine. It was literally the first thing that you know I started working on when I started dabbling around um with with Linear um now probably 5 and 1/2 years ago. Uh and we sort of spent uh an outrageous amount of time you know building out the sync engine as a startup um and usually that's not something that you know any startup would would um usually do. Like you would you know want to build out product first um in order to validate
On Hacker News
You may not need CRDT per-se, but building a collaborative experience is so difficult. I worked on collaborative systems for a bit, and also have read a bit about how Figma and Notion do it (this is a good read: [LINK] ) -- it's still super hard to get right. This talk by Karri about Linear's "sync engine" is also a good watch: [VIDEO] . — saqadri, 2023-11
This article has been quite the inspiration for many projects and progress on this front. I think we're seeing more and more developments around CRDTs and local-first frameworks / applications. I'm working on a few projects in this area: - [LINK] - Notion meets Notebook-style live programming for TypeScript / React - [LINK] - a rich text editor built on TipTap / Prosemirror that supports Yjs for … — YousefED, 2023-10
> Previous sync engines really want you to sync all data Linear had to do all sorts of shenanigans to be able to sync all data, for orgs with lots of it – there's a talk on that here: [VIDEO] — probabletrain, 2025-03
(this should probably be a post) CRDTs and local first are ideas that is perpetually in the hype cycle for the last decade or so, starting around the time Riak CRDTs became a thing and continuing all the way to today. Niki's post is a perfect illustration: CRDTs offer this "magical" experience that seems perfectly good until you try building a product with them. Then it becomes a nightmare of tra… — dgroshev, 2024-06
Heads-up: This is a video ( [VIDEO] ) embedded in a blog post; there doesn't seem to be an accompanying text-only version. — notamy, 2023-06
Related videos
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47
- The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019 · GOTO Conferences · 42:02
- "Simple Made Easy" - Rich Hickey (2011) · Strange Loop Conference · 1:01:38
- What the heck is the event loop anyway? | Philip Roberts | JSConf EU · JSConf · 26:52
- Rich Harris - Rethinking reactivity · You Gotta Love Frontend · 36:45
- "Making Impossible States Impossible" by Richard Feldman · elm-conf · 25:06
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- How To Design A Good API and Why it Matters · Google TechTalks · 1:00:19
- The Value of Values with Rich Hickey · InfoQ · 31:43
- Overwatch Gameplay Architecture and Netcode · GDC Festival of Gaming · 1:04:03
- Solving Problems the Clojure Way - Rafal Dittwald · Clojure/north · 1:02:26
- Lecture 1A: Overview and Introduction to Lisp · MIT OpenCourseWare · 1:12:55
- Alan Kay - Normal Considered Harmful · powerfuloutlet · 1:07:26
- Crockford on JavaScript - Volume 1: The Early Years · YUI Library · 1:42:08
- RailsConf 2016 - Turbolinks 5: I Can’t Believe It’s Not Native! by Sam Stephenson · Confreaks · 37:37
- A Philosophy of Software Design | John Ousterhout | Talks at Google · Talks at Google · 1:01:40