How to Speed up a Python Program 114,000 times.
Channel:InfoQ · Length:59:11 · Published:2012-12-05 · Views:170646
Summary
David Schachter describes how he made a data-collection Python program run 114,000 times faster, drawing on decades of experience. The talk covers pipelining, cache effects and other optimization lessons.
Why HN recommends it
A favorite tech talk on HN for showing that hardware knowledge and generalist skills beat simply adding machines.
Humour, serious technical insight and a good reminder of why being a generalist is an advantage. — HN, 2016
Python, Ruby & scripting languages · Software performance & optimization
Subtitles (excerpt)
[Music] the uh structure of tonight's presentation is pretty straightforward I'm going to talk about the problem that we had the algorithm that was chosen to solve the uh to implement the solution but it ran too slowly and then two different sets of solutions the first set we did in 2011 the second set in 2012 I say we mostly because I have tapeworm hey it's a transparent Universe now you can't hide anything my doctor is pretty upset because I'm the only medical record in his office Still on paper everything else is computerized now I said I know how this works keep me on paper uh then we'll talk about future steps more things we could uh do to speed up the program and the second half of this presentation is going to be Lessons Learned I always think it's worthwhile to spend a few minutes and learn from your mistakes so you can make them again the problem that we're faced with is a fairly straightforward a common problem in these sorts of Enterprises we sell as I said games we don't really sell them we mostly give them away for free and our games are very fun we have a huge pile of people at our company who know how to make things that are fun we're an entertainment firm it's what we do internally we have a
On Hacker News
"How to Speed up a Python Program 114,000 times." - [VIDEO] Humour, serious technical insight and a good reminder of why being a generalist is an advantage. — anoother, 2016-10
I thought this was going to be "How to Speed up a Python Program 114,000 times"[0]. It's an hour-long talk, but quite enjoyable. In the example in the talk, it was less about knowing cs algorithms and more about knowing the details of how modern computer architecture works, but the lesson is the same: sometimes the background knowledge of a cs degree is useful. He also has the same point about op… — lkbm, 2020-10
This will technically only work on pipelined architectures of which all major CPU architectures are one. You will be able to see a significant boost to performance given the following are all true: * You have no inter dependancy of information across the lines * All your data is in the cache * You don't have instructions generated between the two lines that will require some sort of operand forwa… — gravypod, 2016-12
> I remember I recently saw a lecture by an industry professional who took a Python algorithm and improved it 100 000x, in parts. That is, in stages he explained how to utilize the hardware on that particular machine, to improve the algorithm so much that it was hard to believe. Anyone have a link? I suspect it's this one: [VIDEO] — lmc, 2022-05
An example of "scaling in" is this talk, "How to speed up a Python program 114,000 times": [VIDEO] — dmoney, 2015-01
Related videos
- Locksmith ripoffs: Hidden camera investigation (Marketplace) · CBC News · 20:46
- "Games as a service" is fraud. · Accursed Farms · 1:15:54
- Rant: Entity systems and the Rust borrow checker ... or something. · Jonathan Blow · 1:00:52
- Design Microservice Architectures the Right Way · InfoQ · 48:29
- Mike Monteiro: F*ck You, Pay Me · CreativeMornings HQ · 37:53
- Crockford on JavaScript - Volume 1: The Early Years · YUI Library · 1:42:08
- Homemade Transistor Demonstration · Jeri Ellsworth · 3:10
- Where Do Trees Get Their Mass? · Veritasium · 4:10
- Servo Architecture: Safety and Performance · linux conf au 2017 - Hobart, Australia · 45:05
- Email Hates the Living · m · 48:57
- Ruby Conf 12 - Y Not- Adventures in Functional Programming by Jim Weirich · Confreaks · 53:21
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework · Видео с мероприятий {speach! · 49:55
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47
- What the heck is the event loop anyway? | Philip Roberts | JSConf EU · JSConf · 26:52
- CppCon 2015: Andrei Alexandrescu “std::allocator...” · CppCon · 1:12:27