Bringing the Unix Philosophy to Big Data by Bryan Cantrill: FutureStack13

Channel:New Relic · Length:31:08 · Published:2013-11-12 · Views:17568

Watch on YouTube

Summary

Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.

Why HN recommends it

HN recommends it as an elegant argument for simple Unix tools replacing heavyweight Hadoop-style pipelines.

The Bryan Cantrill talk is definitely worth your time — HN, 2015

Databases & storage · Operating systems & kernels · Data science & statistics

Subtitles (excerpt)

thank you very much um good morning thanks for for making it in here uh so normally at any given conference um I'm always reaching back into the Crypt and dropping history on everybody um which I I kind of I kind of like that role um you know some people find it a little bit annoying but I like history um and i' I've kind of come to cherish it as my own that at any given conference I'm the one that's going to be talking about history all Hillary Mason goes 19th century on my ass um so I I I was not ready for that at all I thought that was a terrific terrific talk by Hillary obviously so um I I guess I'm no longer the history guy because I I mean this is not 19th century we're going to talk about about the 20th century um and in particular about bringing the Unix philosophy to Big Data um so first people know what Unix is Right show of hands Unix like that term rings a bell good um so um Unix first appeared actually first started to be written in 1969 um so over 40 years ago well over 40 years ago um and and started to appear slowly in in the 1970s and Unix was was an incredibly uh important development because it was not just a new system but it actually represented a new way of thinking about systems and operating systems

On Hacker News

This article echoes a talk Bryan Cantrill gave two years ago: [VIDEO] It's about how Joyent took the concept of a UNIX pipeline as a true powertool and built a distributed version atop an object filesystem with some little map/reduce syntactic sugar to replace Hadoop jobs with pipelines. The Bryan Cantrill talk is definitely worth your time, but you can get an understanding of Manta with their 3m… — notpeter, 2015-01
They're good points, but from the philosophical point The Unix Way , systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side. I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: [V… — erhardm, 2014-10
1.) Be curious. Stay curious. Experiment, play with things. Your time during university is likely the best time for this and it will likely get harder later on when you're on the job and there's much more emphasis on finished work than on learning experiences. Also, I'm quite sure that it will be beneficial for you if you familiarize yourself with different ways of thinking (obviously programming… — btschaegg, 2022-01

Related videos