Shared memory parallelism in Julia | Kiran Pamnany | Cambridge Julia Meetup (May 2018)
Channel:The Julia Programming Language · Length:26:33 · Published:2018-05-17 · Views:6792
Summary
Kiran Pamnany's Cambridge Julia Meetup talk on Julia's shared memory parallelism and its new task scheduler. It argues that fork-join and work-stealing models nest poorly and presents depth-first scheduling for composable parallelism.
Why HN recommends it
Recommended in Julia threading threads as an insightful, simple explanation of the new task scheduler.
I found his talk to be an insightful overview of this work — HN, 2018
Python, Ruby & scripting languages · Software performance & optimization · Operating systems & kernels
Subtitles (excerpt)
okay everybody hear me okay I imagine so okay so shared memory parallel ISM and Julia if anybody was at Julia Khan a couple years ago you may have seen some conversation about this we've been working at it for a while I'm gonna start I mean this is a very basic question but you know the question does arise why threads are alway why can't we just do everything with processes and messaging and that's a reasonable question in in the past but no longer so because folks may have heard of the Knights processors and greater than 200 threads available on a single processor at times like this you're a memory poor core reduces to the point where you cannot literally cannot do anything useful with the process per core or a hardware thread so you pretty much need threads which is essentially shared memory parallel ISM and that gives you typically quicker performance less communication requirement and you reduced memory needed by your application but of course as we often know shared memory parallelism always opens you up to a vast host of dangerous programming errors which is sort of the cost of doing business so again early principles will partition parallelization according to these two broad domains there are many other ways to categorize things
On Hacker News
By far the most interesting part of this release is the new multi-threading features, I recommend reading [LINK] for an overview, and watching [VIDEO] for a talk about some unique stuff that's being worked on with Julia's multi-threading (though the depth-first scheduling is not in 1.3 afaik). — falkaer, 2019-11
Consider a machine with N cores running a program which spawns N parallel tasks, each of which in turn spawns N parallel tasks. Which subtasks do the cores work on initially? If it's the first subtask of each top-level task, that's breadth-first scheduling. If it's all N subtasks of the first top-level task, that's depth-first scheduling. The latter is better for high-performance computing becaus… — StefanKarpinski, 2019-07
This work has been a monumental collaborative effort and I think the eventual outcome will be pretty spectacular. I gave a talk at an MIT Julia meetup a few months ago that the author of this PR also spoke at; I found his talk to be an insightful overview of this work: [VIDEO] — jrevels, 2018-10
checkout [VIDEO] which explains the new features of the task scheduler in a simple fashion. Basically Julia is employing a new algorithm to schedule tasks, where traditional methods would get stuck/blocked. Essentially, if you run a job with parallel tasks and one of the threads finishes, it's extremely difficult to schedulate the free thread, since the data may still be used by the other threads… — boromi, 2018-10
Does the Rust Rayon library have the issue the speaker was describing, where parallel inner loops mostly get executed serially once an outer loop is parallelized? (I assume it does, from its Cilk heritage?) Are there workarounds for when that is a problem? Edit: Dumb mistake on my part. I was looking at the video linked in the top comment ( [VIDEO] ) and then I came back to this thread and though… — oconnor663, 2019-11
Related videos
- Making R run Fast - Jan Vitek · John O'Neill · 1:21:38
- How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework · Видео с мероприятий {speach! · 49:55
- The Unreasonable Effectiveness of Multiple Dispatch | Stefan Karpinski | JuliaCon 2019 · The Julia Programming Language · 35:24
- Ruby Conf 12 - Y Not- Adventures in Functional Programming by Jim Weirich · Confreaks · 53:21
- The spelled-out intro to neural networks and backpropagation: building micrograd · Andrej Karpathy · 2:25:52
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47
- What the heck is the event loop anyway? | Philip Roberts | JSConf EU · JSConf · 26:52
- The Tragedy of systemd · linux.conf.au · 47:18
- CppCon 2015: Andrei Alexandrescu “std::allocator...” · CppCon · 1:12:27
- How To Design A Good API and Why it Matters · Google TechTalks · 1:00:19
- The Problem with Time & Timezones - Computerphile · Computerphile · 10:12
- Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc) · DevGAMM · 1:02:41
- "Performance Matters" by Emery Berger · Strange Loop Conference · 42:15
- RailsConf 2016 - Turbolinks 5: I Can’t Believe It’s Not Native! by Sam Stephenson · Confreaks · 37:37
- The Thirty Million Line Problem · Molly Rocket · 1:48:55
- CppCon 2017: Matt Kulukundis “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step” · CppCon · 59:33