AWS re:Invent 2017: How Netflix Tunes Amazon EC2 Instances for Performance (CMP325)

Channel:Amazon Web Services · Length:56:54 · Published:2017-12-13 · Views:18438

Watch on YouTube

Summary

A Netflix performance engineer presents how Netflix tunes EC2 instances, covering instance type choice, Xen modes, SR-IOV and kernel tuning. The goal is near bare-metal performance in the cloud.

Why HN recommends it

HN cites it as an example of workload-dependent performance tuning at scale and of Netflix's AWS practices.

performance tuning is an art form as it is workload dependant — HN, 2019

Software performance & optimization · Operating systems & kernels · Cloud, DevOps & reliability

Subtitles (excerpt)

g'day my name is brynan Gregg and welcome to my talk I work at Netflix and we're deployed on AWS for our cloud it's very large you may have heard of it and that's running Ubuntu Linux we also have a CDN that's based on free D FreeBSD that's physical service Netflix is an awesome place to work and the team I'm on is the performance and operating systems team our job is to evaluate technology and times like reinvent keep us all very busy with all the new announcements we come up with recommendations and best practices for the Netflix cloud as part of the work that we do engineering work we often have to create new tools especially observability tools to fill in missing gaps and we will often open-source those tools we help with projects new rollouts of technologies and applications and we can also do Incident Response if there is a performance issue my manager at hunter is in the room is here reinvent and we're actually hiring a couple of positions so you can come join me and work on my team at Netflix we'll all be at the booth after this talk the Netflix booth in this talk I'm going to go through how we do performance tuning of ec2 at Netflix I give many talks where I'm covering what I do with kernel analysis

On Hacker News

it was actually 5x, my bad, but here's the references: Original info being mentioned as a rumour (2016): [LINK] Slides for confirmation by Greeg (2017): [LINK] Talk (2017): [VIDEO] More discussion: [LINK] Original tweet has been deleted, but I have a screenshot here slide 9: [LINK] — dijit, 2022-05
Well, most distributions choose saner defaults. For instance, RHEL/CentOS/AmazonLinux use noop io-scheduler by default for SSD based machines. Though, I get your point: for instance, the TCP keep-alive defaults aren't suitable for all kinds of workloads: [LINK] I guess, that's why performance tuning is an art form as it is workload dependant: [VIDEO] — ignoramous, 2019-03
They use a lot of AWS services - not just virtual machines - First they use Ubuntu Linux - SES (Email) - ElasticSearch (AWS has their own managed version) - SQS (queueing system) - S3 (storage) -auto scaling Most of the optimization's they do are standard things at scale where they are tuning by measuring performance. [VIDEO] and the slide deck from the presentation. [LINK] But that's even more o… — scarface74, 2018-05

Related videos