Rust at speed — building a fast concurrent database
Channel:Jon Gjengset · Length:52:54 · Published:2019-01-05 · Views:225927
Summary
A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.
Why HN recommends it
HN points to it as a detailed, captivating talk by the Noria author, including a good explanation of deduplicating data between maps using unsafe.
gave a great talk about the system — HN, 2021
C, C++, Rust, Go & systems languages · Software performance & optimization · Databases & storage
Subtitles (excerpt)
hi welcome everybody I'm Larry Rudolph from the labs team part of the goal the Lapps team is to reach out to academia and this is a special talk I met John at csail when we were when Trevor and I were there talking about security of firmware and we had this idea that we should use rust and so we met so there's the MIT rest expert or the Northeast rustic spirit or the worldwide rust expert I don't know which one call yourself with but his research is also on databases novel ways databases and coordination and conflicts and stuff that I really like so when inviting to give a talk I said you have to talk about both things and John said no problem I guess he comes here from I guess you had undergrad in Australia and then you went to London and I was from Norway you know he went the wrong way and then finally sort of came back up and ended up at MIT and he's a grad student at my Tec cell but let's so he's going to give a talk and then there's be a deep dive or extended Q&A if you want to stay and ask but let's make him welcome thank you so I'm John thanks for the introduction I'm here to talk to you about rust in general and in particular about a research system that I've been building at MIT
On Hacker News
The author of Noria, Jon Gjengset, gave a great talk about the system: [VIDEO] The speeds achieved are impressive but you do not get consistency. — Orou, 2021-12
Anecdotal - One of favorite YouTube live-streaming personalities, Jon Gjengset[1] is a Ph.D student at MIT CSAIL. His talks are very detailed[4] and captivating[3] [1] [LINK] Talks [2] [VIDEO] [3] [LINK] Rust Details(my favorites) [4] [LINK] — harikb, 2020-06
:wave: Author of the paper this work is based on here. I'm so excited to see dynamic, partially-stateful data-flow for incremental materialized view maintenance becoming more wide-spread! I continue to think it's a _great_ idea, and the speed-ups (and complexity reduction) it can yield are pretty immense, so seeing more folks building on the idea makes me very happy. The PlanetScale blog post ref… — Jonhoo, 2022-11
He explains this at roughly the 40 minute mark. [1] "There's 7 more lines of `unsafe` that avoids keeping both maps. You still keep two maps, but you de-duplicate the data between those two maps." [VIDEO] — dpezely, 2019-01
Related videos
- Crust of Rust: Lifetime Annotations · Jon Gjengset · 1:33:23
- How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework · Видео с мероприятий {speach! · 49:55
- CppCon 2015: Andrei Alexandrescu “std::allocator...” · CppCon · 1:12:27
- Lexical Scanning in Go - Rob Pike · GoogleFOSSSydney · 51:17
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- The Value of Values with Rich Hickey · InfoQ · 31:43
- Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc) · DevGAMM · 1:02:41
- "Performance Matters" by Emery Berger · Strange Loop Conference · 42:15
- The Thirty Million Line Problem · Molly Rocket · 1:48:55
- CppCon 2017: Matt Kulukundis “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step” · CppCon · 59:33
- AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) · Amazon Web Services · 59:56
- RustConf 2018 - Closing Keynote - Using Rust For Game Development by Catherine West · Rust Programming Language · 42:29
- Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp · Erlang Solutions · 49:15
- How Crash Bandicoot Hacked The Original Playstation | War Stories | Ars Technica · Ars Technica · 31:37
- "Turning the database inside out with Apache Samza" by Martin Kleppmann · Strange Loop Conference · 47:42
- "How NOT to Measure Latency" by Gil Tene · Strange Loop Conference · 42:59