FreeBSD optimizations used by Netflix serving at 800Gb/s - Drew Gallatin - EuroBSDcon 2022

Channel:EuroBSDCon · Length:44:04 · Published:2022-10-27 · Views:27848

Watch on YouTube

Summary

Netflix engineer Drew Gallatin's EuroBSDcon 2022 talk quantifies the older FreeBSD optimizations that let Netflix serve web traffic at 800Gb/s, including TSO, LRO, VM page cache zones, asynchronous sendfile and unmapped mbufs.

Why HN recommends it

HN points to it as the recorded talk behind the Netflix 800Gb/s slides and as a deep look at kernel-level optimizations for high throughput.

this might be a little lower level that you may be interested in as it relates to kernel optimizations to jack bit throughput rates — HN, 2023

Software performance & optimization · Operating systems & kernels · Networking & internet infrastructure

Subtitles (excerpt)

so I'm Drew Gallatin uh I do FreeBSD performance work for our CDN servers at Netflix and usually like it talks about how to make things better but today I'm going to give a talk about basically what breaks when it disables certain optimizations basically how do I make things worse so just to give you a bit of background um there's 800 gigs in the title and we will get to 800 gigs and we got to 800 gigs last fall just after the last year of ESD con um and I and and that kind of motivated me to look back and see like how much the various optimizations that that we and the FreeBSD Community have made over the years have helped us so I want to emphasize the FreeBSD Community because most of the things I'm talking about today weren't done at Netflix most of them were done by the great people in the community um that are some of which are here at this at this conference and the other thing you're going to notice is that there's only one optimization between you and no chaos a lot of a lot of times so to give a background to what we do on our CDN servers we run FreeBSD current We Run The the nginx web server and we have the easiest job in the world because we just serve static files via send file uh this is

On Hacker News

OP mentions that "I would love to be a little mouse and peek at YouTube’s complete architecture to see how far we are from them." You can occasionally find posts -often linked here- from another player in streaming video which you might have heard of, discussing technical architecture. For example, this might be a little lower level that you may be interested in as it relates to kernel optimizati… — thomasjudge, 2023-10
This talk was given at this years EuroBSDcon in Vienna, recording is up on YouTube. [LINK] [VIDEO] Some really great talks this year from all the *BSDs, highly recommend checking a look: [LINK] — notaplumber1, 2022-11
Netflix discusses their reasons starting at 18:20: [LINK] tl;dw: the performance, the efficiency of development, the community, FreeBSD is a complete operating system, the code base is smaller, the ports system, and the license. and this video covers the optimizations Netflix has made to FreeBSD: [VIDEO] Also potentially a reason: According to drewg123, Linux's kTLS was broken. Which I see drewg1… — craftkiller, 2025-07
> Ethernet is stuck on 1500 byte frames, with the result of limiting TCP as well. That's just nuts on 100G or 400G networks, even on 10G it's a bit of a stretch already. How limiting is 1500 for the end-nodes for most people? For access-, distribution-, and core-layer switches the ASICs process things at wire speed, so does it matter what the frame size is? I can see some server applications, but… — throw0101c, 2023-06
> How did you generate those flamegraphs and what other tools did you use to measure performance? We have an internal shell script that takes hwpmc output and generates flamegraphs from the stacks. It also works with dtrace. I'm a huge fan of dtrace. I also make heavy use of lockstat, AMD uProf, and Intel Vtune. > Did the graph show the bottleneck contention on aio queue? Did the graph show that … — drewg123, 2022-11

Related videos