Larry Hastings - Removing Python's GIL: The Gilectomy - PyCon 2016

Channel:PyCon 2016 · Length:32:07 · Published:2016-06-01 · Views:48693

Watch on YouTube

Summary

Larry Hastings explains what Python's GIL is, what problems it solves and how it works, and presents his Gilectomy project to remove it, including the performance and C-extension obstacles.

Why HN recommends it

The default reference in Python performance threads for why removing the GIL is hard.

I think Larry Hasting's first Gilectomy talk was extremely insightful — HN, 2017

Python, Ruby & scripting languages · Software performance & optimization · Operating systems & kernels

Subtitles (excerpt)

[applause] (Larry Hastings) Thank you. Yes, this talk has changed. If you're looking for my talk about the GIL, this isn't it. This is a talk about removing Python's GIL. And thank you all for practicing getting your planes for later tonight. So, as a preface for this talk, I want to warn you: this talk is going to be exceedingly technical. I was encouraged to make this talk very technical, actually, by the program committee. So I'm going to talk about a lot of technical topics and I don't really have that much time to introduce them. So, I'm assuming that you're pretty familiar or comfortable at least with me talking about multithreading and about CPython internals from a very high level. I'm not going to be digging way down, but you have to understand kind of the basics of CPython internals, like reference counting. If you're not familiar with that stuff, a good refresher course or an introduction would be my previous talk, called "Python's Infamous GIL", which, again, I was supposed to give here, but then we switched it. There's videos of that online. So let's start with a little history about the GIL. The GIL was added in 1992. Guido wanted to add support for this new thing that operating systems are starting to do, called threading.

On Hacker News

I feel like the GIL is, at this point, Python's most infamous attribute. For a long time I thought it was also the biggest flaw with Python...but over time I care less and less about it. I think the first thing to realize is that single-threaded performance is often significantly better with the GIL than without it. I think Larry Hasting's first Gilectomy talk was extremely insightful (about the … — gshulegaard, 2017-08
There's a great talk by David Beazley (I don't remember which one) that he basically explains that removing the GIL isn't difficult at all. After all, it's just one lock. The issue is all the libraries and packages that were built around it. It's now been YEARS of building on top of the GIL. So anyways, PEP 703 is a a nice effort, but I doubt we (everyday mortals) can enjoy it. Meta, and big comp… — santiagobasulto, 2023-07
[VIDEO] Or search “gilectomy” on LWN. Or check out stackless etc.; it turns out removing the GIL technically is historically one of the easier parts of removing the GIL entirely. It sounds like your main problem is with python’s GC model. Reference counting doesn’t have to be thread-unsafe, but in scripting/interpreted-ish languages that want their threading story to involve seamless (I.e. no spe… — zbentley, 2018-10
Larry Hastings - Removing Python's GIL: The Gilectomy - PyCon 2016 [VIDEO] (I'm pretty sure this is the video I'm thinking of) It's 30m, but worth it if you're interested. Not sure what progress has been made since then. — pfranz, 2017-03
See Larry Hastings' talk "Removing Python's GIL: The Gilectomy" from PyCon 2016. [VIDEO] As well as his earlier talk "Python's Infamous GIL". [LINK] The GIL is a simple design that provides serious benefit to Python (end user) developers. There are plenty of options for working around this for parallelism, such as the built-in module multiprocessing. — tedmiston, 2016-07

Related videos