Rust at speed — building a fast concurrent database

Channel:Jon Gjengset · Length:52:54 · Published:2019-01-05 · Views:225927

Watch on YouTube

Summary

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

Why HN recommends it

HN points to it as a detailed, captivating talk by the Noria author, including a good explanation of deduplicating data between maps using unsafe.

gave a great talk about the system — HN, 2021

C, C++, Rust, Go & systems languages · Software performance & optimization · Databases & storage

Subtitles (excerpt)

hi welcome everybody I'm Larry Rudolph from the labs team part of the goal the Lapps team is to reach out to academia and this is a special talk I met John at csail when we were when Trevor and I were there talking about security of firmware and we had this idea that we should use rust and so we met so there's the MIT rest expert or the Northeast rustic spirit or the worldwide rust expert I don't know which one call yourself with but his research is also on databases novel ways databases and coordination and conflicts and stuff that I really like so when inviting to give a talk I said you have to talk about both things and John said no problem I guess he comes here from I guess you had undergrad in Australia and then you went to London and I was from Norway you know he went the wrong way and then finally sort of came back up and ended up at MIT and he's a grad student at my Tec cell but let's so he's going to give a talk and then there's be a deep dive or extended Q&A if you want to stay and ask but let's make him welcome thank you so I'm John thanks for the introduction I'm here to talk to you about rust in general and in particular about a research system that I've been building at MIT

On Hacker News

The author of Noria, Jon Gjengset, gave a great talk about the system: [VIDEO] The speeds achieved are impressive but you do not get consistency. — Orou, 2021-12
Anecdotal - One of favorite YouTube live-streaming personalities, Jon Gjengset[1] is a Ph.D student at MIT CSAIL. His talks are very detailed[4] and captivating[3] [1] [LINK] Talks [2] [VIDEO] [3] [LINK] Rust Details(my favorites) [4] [LINK] — harikb, 2020-06
:wave: Author of the paper this work is based on here. I'm so excited to see dynamic, partially-stateful data-flow for incremental materialized view maintenance becoming more wide-spread! I continue to think it's a _great_ idea, and the speed-ups (and complexity reduction) it can yield are pretty immense, so seeing more folks building on the idea makes me very happy. The PlanetScale blog post ref… — Jonhoo, 2022-11
He explains this at roughly the 40 minute mark. [1] "There's 7 more lines of `unsafe` that avoids keeping both maps. You still keep two maps, but you de-duplicate the data between those two maps." [VIDEO] — dpezely, 2019-01

Related videos