Larry Hastings - Removing Python's GIL: The Gilectomy - PyCon 2016
Channel:PyCon 2016 · Length:32:07 · Published:2016-06-01 · Views:48693
Summary
Larry Hastings explains what Python's GIL is, what problems it solves and how it works, and presents his Gilectomy project to remove it, including the performance and C-extension obstacles.
Why HN recommends it
The default reference in Python performance threads for why removing the GIL is hard.
I think Larry Hasting's first Gilectomy talk was extremely insightful — HN, 2017
Python, Ruby & scripting languages · Software performance & optimization · Operating systems & kernels
Subtitles (excerpt)
[applause] (Larry Hastings) Thank you. Yes, this talk has changed. If you're looking for my talk about the GIL, this isn't it. This is a talk about removing Python's GIL. And thank you all for practicing getting your planes for later tonight. So, as a preface for this talk, I want to warn you: this talk is going to be exceedingly technical. I was encouraged to make this talk very technical, actually, by the program committee. So I'm going to talk about a lot of technical topics and I don't really have that much time to introduce them. So, I'm assuming that you're pretty familiar or comfortable at least with me talking about multithreading and about CPython internals from a very high level. I'm not going to be digging way down, but you have to understand kind of the basics of CPython internals, like reference counting. If you're not familiar with that stuff, a good refresher course or an introduction would be my previous talk, called "Python's Infamous GIL", which, again, I was supposed to give here, but then we switched it. There's videos of that online. So let's start with a little history about the GIL. The GIL was added in 1992. Guido wanted to add support for this new thing that operating systems are starting to do, called threading.
On Hacker News
I feel like the GIL is, at this point, Python's most infamous attribute. For a long time I thought it was also the biggest flaw with Python...but over time I care less and less about it. I think the first thing to realize is that single-threaded performance is often significantly better with the GIL than without it. I think Larry Hasting's first Gilectomy talk was extremely insightful (about the … — gshulegaard, 2017-08
There's a great talk by David Beazley (I don't remember which one) that he basically explains that removing the GIL isn't difficult at all. After all, it's just one lock. The issue is all the libraries and packages that were built around it. It's now been YEARS of building on top of the GIL. So anyways, PEP 703 is a a nice effort, but I doubt we (everyday mortals) can enjoy it. Meta, and big comp… — santiagobasulto, 2023-07
[VIDEO] Or search “gilectomy” on LWN. Or check out stackless etc.; it turns out removing the GIL technically is historically one of the easier parts of removing the GIL entirely. It sounds like your main problem is with python’s GC model. Reference counting doesn’t have to be thread-unsafe, but in scripting/interpreted-ish languages that want their threading story to involve seamless (I.e. no spe… — zbentley, 2018-10
Larry Hastings - Removing Python's GIL: The Gilectomy - PyCon 2016 [VIDEO] (I'm pretty sure this is the video I'm thinking of) It's 30m, but worth it if you're interested. Not sure what progress has been made since then. — pfranz, 2017-03
See Larry Hastings' talk "Removing Python's GIL: The Gilectomy" from PyCon 2016. [VIDEO] As well as his earlier talk "Python's Infamous GIL". [LINK] The GIL is a simple design that provides serious benefit to Python (end user) developers. There are plenty of options for working around this for parallelism, such as the built-in module multiprocessing. — tedmiston, 2016-07
Related videos
- Understanding the Python GIL · David Beazley · 46:09
- Complete Apple WWDC | Steve Jobs talk and answer developers questions | 1997 · TheAppleFanBoy - Apple & Computer Archives · 1:11:16
- Ruby Conf 12 - Y Not- Adventures in Functional Programming by Jim Weirich · Confreaks · 53:21
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework · Видео с мероприятий {speach! · 49:55
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47
- What the heck is the event loop anyway? | Philip Roberts | JSConf EU · JSConf · 26:52
- The Tragedy of systemd · linux.conf.au · 47:18
- CppCon 2015: Andrei Alexandrescu “std::allocator...” · CppCon · 1:12:27
- How To Design A Good API and Why it Matters · Google TechTalks · 1:00:19
- The Problem with Time & Timezones - Computerphile · Computerphile · 10:12
- Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc) · DevGAMM · 1:02:41
- "Performance Matters" by Emery Berger · Strange Loop Conference · 42:15
- RailsConf 2016 - Turbolinks 5: I Can’t Believe It’s Not Native! by Sam Stephenson · Confreaks · 37:37
- The Thirty Million Line Problem · Molly Rocket · 1:48:55
- CppCon 2017: Matt Kulukundis “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step” · CppCon · 59:33