Lecture 1: Introduction to Information Theory

Channel:Jakob Foerster · Length:1:01:51 · Published:2014-04-26 · Views:413411

Watch on YouTube

Summary

The first lecture of David MacKay's Cambridge course on information theory, pattern recognition and neural networks. It introduces the core ideas of his book Information Theory, Inference, and Learning Algorithms.

Why HN recommends it

Recommended as an enjoyable, clear introduction to information theory that pairs well with MacKay's free textbook.

It was very enjoyable for me to learn the material from his lectures. — HN, 2020

Machine learning & deep learning · Information theory & signal processing · University lectures & courses

Subtitles (excerpt)

Okay, welcome everyone. Thank you very much for coming to this, the first lecture on a course called information theory, pattern recognition, and neural networks. This first lecture is an introduction to information theory. Information theory was invented by Claude Shannon to solve communication problems. And the fundamental problem that Shannon defined was the task of reliable communication over an unreliable channel. So, that's our fundamental problem, and I'll give some examples right now. So, some examples of channels. One channel we're using right now is the channel that goes from my voice to your ear, and the medium is air. Another channel that we may be using is a channel that goes from our eyes up to other bits of our nervous system. And there the medium is sort of bags of cytoplasm and electrical implant impulses. Another example of a channel that we're all carrying around with us is DNA. We've got about um 10 to the 13 copies of our genome on board us in each of our 10 to the 13 cells and every time one of those cells replicates, we send two copies of the genome into those cells. Repeat that 40 times and you've gone from an egg to a human being and maybe onto the next generation. And another example is the

On Hacker News

It‘s such a tragedy that he died so young. Prof Mackay was a role model who impacted our world in many positive ways (e.g. his work in information theory, Bayesian statistics, environmental science). I recommend his lecture on information theory which is available on youtube [1]. He also documented the progress of his cancer very detailed on his blog [2], which is a very interesting (and sad) rea… — samch93, 2021-10
Had . Unfortunately Professor MacKay passed away from cancer in 2016. We lost a great soul. I concur that his textbook and sense of humor were fantastic. It was very enjoyable for me to learn the material from his lectures. They are a great supplement to the textbook (or maybe vice versa?) and you can find them here: [VIDEO] — maxov, 2020-06
I recommend David MacKay's Intro to Info Theory class: [VIDEO] There is a free book for it at: [LINK] Daphne Koller's 3 coursara classes on Probabilistic Graphical Models are also really good. — swframe2, 2018-02
Thanks for the list. Interesting to see that the book by the late David MacKay made it to the list, it is available to download in several formats from the book's website. Information Theory, Inference, and Learning Algorithms: [LINK] The lectures based on the book by David MacKay himself: [VIDEO] Probably it's not a surprise that the father of information theory Claude Shannon is the earliest on… — teleforce, 2024-05
"Your browser does not support playback of available video formats. Please install Adobe Flash player or upgrade to a more modern browser." ...Okay? I found the videos on youtube as well: [VIDEO] — noud, 2023-02

Related videos