FreeBSD optimizations used by Netflix serving at 800Gb/s - Drew Gallatin - EuroBSDcon 2022
Channel:EuroBSDCon · Length:44:04 · Published:2022-10-27 · Views:27848
Summary
Netflix engineer Drew Gallatin's EuroBSDcon 2022 talk quantifies the older FreeBSD optimizations that let Netflix serve web traffic at 800Gb/s, including TSO, LRO, VM page cache zones, asynchronous sendfile and unmapped mbufs.
Why HN recommends it
HN points to it as the recorded talk behind the Netflix 800Gb/s slides and as a deep look at kernel-level optimizations for high throughput.
this might be a little lower level that you may be interested in as it relates to kernel optimizations to jack bit throughput rates — HN, 2023
Software performance & optimization · Operating systems & kernels · Networking & internet infrastructure
Subtitles (excerpt)
so I'm Drew Gallatin uh I do FreeBSD performance work for our CDN servers at Netflix and usually like it talks about how to make things better but today I'm going to give a talk about basically what breaks when it disables certain optimizations basically how do I make things worse so just to give you a bit of background um there's 800 gigs in the title and we will get to 800 gigs and we got to 800 gigs last fall just after the last year of ESD con um and I and and that kind of motivated me to look back and see like how much the various optimizations that that we and the FreeBSD Community have made over the years have helped us so I want to emphasize the FreeBSD Community because most of the things I'm talking about today weren't done at Netflix most of them were done by the great people in the community um that are some of which are here at this at this conference and the other thing you're going to notice is that there's only one optimization between you and no chaos a lot of a lot of times so to give a background to what we do on our CDN servers we run FreeBSD current We Run The the nginx web server and we have the easiest job in the world because we just serve static files via send file uh this is
On Hacker News
OP mentions that "I would love to be a little mouse and peek at YouTube’s complete architecture to see how far we are from them." You can occasionally find posts -often linked here- from another player in streaming video which you might have heard of, discussing technical architecture. For example, this might be a little lower level that you may be interested in as it relates to kernel optimizati… — thomasjudge, 2023-10
This talk was given at this years EuroBSDcon in Vienna, recording is up on YouTube. [LINK] [VIDEO] Some really great talks this year from all the *BSDs, highly recommend checking a look: [LINK] — notaplumber1, 2022-11
Netflix discusses their reasons starting at 18:20: [LINK] tl;dw: the performance, the efficiency of development, the community, FreeBSD is a complete operating system, the code base is smaller, the ports system, and the license. and this video covers the optimizations Netflix has made to FreeBSD: [VIDEO] Also potentially a reason: According to drewg123, Linux's kTLS was broken. Which I see drewg1… — craftkiller, 2025-07
> Ethernet is stuck on 1500 byte frames, with the result of limiting TCP as well. That's just nuts on 100G or 400G networks, even on 10G it's a bit of a stretch already. How limiting is 1500 for the end-nodes for most people? For access-, distribution-, and core-layer switches the ASICs process things at wire speed, so does it matter what the frame size is? I can see some server applications, but… — throw0101c, 2023-06
> How did you generate those flamegraphs and what other tools did you use to measure performance? We have an internal shell script that takes hwpmc output and generates flamegraphs from the stacks. It also works with dtrace. I'm a huge fan of dtrace. I also make heavy use of lockstat, AMD uProf, and Intel Vtune. > Did the graph show the bottleneck contention on aio queue? Did the graph show that … — drewg123, 2022-11
Related videos
- Zebras All the Way Down - Bryan Cantrill, Uptime 2017 · Code & Supply · 46:36
- How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework · Видео с мероприятий {speach! · 49:55
- The Tragedy of systemd · linux.conf.au · 47:18
- CppCon 2015: Andrei Alexandrescu “std::allocator...” · CppCon · 1:12:27
- Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc) · DevGAMM · 1:02:41
- "Performance Matters" by Emery Berger · Strange Loop Conference · 42:15
- The Thirty Million Line Problem · Molly Rocket · 1:48:55
- CppCon 2017: Matt Kulukundis “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step” · CppCon · 59:33
- VCF East: Ken Thompson interviewed by Brian Kernighan · Vintage Computer Federation · 1:03:51
- Harder Drive: Hard drives we didn't want or need · suckerpinch · 36:46
- Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp · Erlang Solutions · 49:15
- How Crash Bandicoot Hacked The Original Playstation | War Stories | Ars Technica · Ars Technica · 31:37
- LISA11 - Fork Yeah! The Rise and Development of illumos · USENIX · 1:04:03
- "How NOT to Measure Latency" by Gil Tene · Strange Loop Conference · 42:59
- USENIX ATC '21/OSDI '21 Joint Keynote Address-It's Time for Operating Systems to Rediscover Hardware · USENIX · 1:06:19
- Refterm Lecture Part 1 - Philosophies of Optimization · Molly Rocket · 18:41