Bringing the Unix Philosophy to Big Data by Bryan Cantrill: FutureStack13
Channel:New Relic · Length:31:08 · Published:2013-11-12 · Views:17568
Summary
Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.
Why HN recommends it
HN recommends it as an elegant argument for simple Unix tools replacing heavyweight Hadoop-style pipelines.
The Bryan Cantrill talk is definitely worth your time — HN, 2015
Databases & storage · Operating systems & kernels · Data science & statistics
Subtitles (excerpt)
thank you very much um good morning thanks for for making it in here uh so normally at any given conference um I'm always reaching back into the Crypt and dropping history on everybody um which I I kind of I kind of like that role um you know some people find it a little bit annoying but I like history um and i' I've kind of come to cherish it as my own that at any given conference I'm the one that's going to be talking about history all Hillary Mason goes 19th century on my ass um so I I I was not ready for that at all I thought that was a terrific terrific talk by Hillary obviously so um I I guess I'm no longer the history guy because I I mean this is not 19th century we're going to talk about about the 20th century um and in particular about bringing the Unix philosophy to Big Data um so first people know what Unix is Right show of hands Unix like that term rings a bell good um so um Unix first appeared actually first started to be written in 1969 um so over 40 years ago well over 40 years ago um and and started to appear slowly in in the 1970s and Unix was was an incredibly uh important development because it was not just a new system but it actually represented a new way of thinking about systems and operating systems
On Hacker News
This article echoes a talk Bryan Cantrill gave two years ago: [VIDEO] It's about how Joyent took the concept of a UNIX pipeline as a true powertool and built a distributed version atop an object filesystem with some little map/reduce syntactic sugar to replace Hadoop jobs with pipelines. The Bryan Cantrill talk is definitely worth your time, but you can get an understanding of Manta with their 3m… — notpeter, 2015-01
They're good points, but from the philosophical point The Unix Way , systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side. I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: [V… — erhardm, 2014-10
1.) Be curious. Stay curious. Experiment, play with things. Your time during university is likely the best time for this and it will likely get harder later on when you're on the job and there's much more emphasis on finished work than on learning experiences. Also, I'm quite sure that it will be beneficial for you if you familiarize yourself with different ways of thinking (obviously programming… — btschaegg, 2022-01
Related videos
- Moneyball - Firing people · The BigTime Group · 3:02
- A Better Default Colormap for Matplotlib | SciPy 2015 | Nathaniel Smith and Stéfan van der Walt · Enthought · 19:08
- The Tragedy of systemd · linux.conf.au · 47:18
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- The Value of Values with Rich Hickey · InfoQ · 31:43
- The Thirty Million Line Problem · Molly Rocket · 1:48:55
- VCF East: Ken Thompson interviewed by Brian Kernighan · Vintage Computer Federation · 1:03:51
- AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) · Amazon Web Services · 59:56
- LISA11 - Fork Yeah! The Rise and Development of illumos · USENIX · 1:04:03
- "Turning the database inside out with Apache Samza" by Martin Kleppmann · Strange Loop Conference · 47:42
- USENIX ATC '21/OSDI '21 Joint Keynote Address-It's Time for Operating Systems to Rediscover Hardware · USENIX · 1:06:19
- Is Most Published Research Wrong? · Veritasium · 12:22
- AT&T Archives: The UNIX Operating System · AT&T Tech Channel · 27:27
- Conflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016 · GOTO Conferences · 51:35
- 01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019) · CMU Database Group · 1:06:44
- Unix and Google · FOAM · 7:15