Lecture 1: Introduction to Information Theory

332,841

3,844 0

Published 2014-04-26

Lecture 1 of the Course on Information Theory, Pattern Recognition, and Neural Networks.
Produced by: David MacKay (University of Cambridge)
Author: David MacKay, University of Cambridge

A series of sixteen lectures covering the core of the book "Information Theory, Inference, and Learning Algorithms" (Cambridge University Press, 2003, www.inference.eng.cam.ac.uk/mackay/itila/) which can be bought at Amazon (www.amazon.co.uk/exec/obidos/ASIN/0521642981/david…, and is available free online (www.inference.eng.cam.ac.uk/mackay/itila/).

A subset of these lectures used to constitute a Part III Physics course at the University of Cambridge. The high-resolution videos and all other course material can be downloaded from the Cambridge course website (www.inference.eng.cam.ac.uk/mackay/itprnn/).

Snapshots of the lecture can be found here:
www.inference.eng.cam.ac.uk/itprnn_lectures/

These lectures are also available at
videolectures.net/course_information_theory_patter…
(synchronized with snapshots and slides)

All Comments (21)

@mohamedrabie4663 1 year ago

RIP prof David, you were, and still a great inspiration to us
@iwtwb8 7 years ago

I was saddened to read that David MacKay passed away earlier this year.
@prajwolgyawali6770 4 years ago

Fundamental Problem: Reliable communication over unreliable channel Binary Symmetric Channel: 9:35 Disk Drive Problem: 11:30 Redundancy: 23:00 Repetition Code: 24:35 Decoding is inference Inverse Probability: 31:33 Forward Probability: 40:30 Hamming Code: 47:10 (https://www.youtube.com/watch?v=eixCGqdlGxQ&list=PLJHszsWbB6hqkOyFCQOAlQtfzC1G9sf2_%29) Capacity of channel: 58:10 Shannon Noisy Channel Coding Theorem: 58:45 The Weighing Problem 1:00:50
@AvindraGoolcharan 4 years ago

This may be the longest blackboard I've ever seen
@baganatube 6 years ago

I don't mind the slowness. With some decoding, my brain is receiving a cleaner signal with that extra redundancy.
@oscarbergqvist4992 4 years ago

Thank you for a great lecture, looking forward to follow the rest and study the book!
@dragonfly3139 7 years ago

Thank you for these great lectures your memories will live on ... RIP
@IrfanAli-jl7vb 5 years ago

Thank you, Thank you, Thank you for sharing these excellent video lectures. Dr Mackay is amazing at teaching complicated topics. These lectures are great supplement to his excellent book on information theory which has so many excellent plots and graphs that enables one to visualize information theory. Information theory comes alive in pictures. Thank you for sharing these.
@TheNiro87 2 years ago

This is great, thank you! The lecture is far more entertaining than just reading the book.
@leduran 2 years ago

These lectures are great. Thanks for sharing.
@fireflystar5333 1 year ago

This is a great lecture. The design of case problem is really helpful here. Thanks for the lecture.
@a_user_from_earth 4 months ago

what an amazing lecture and great teacher. May you rest in peace. You will not be forgotten.
@jedrekwrzosek6918 1 year ago

I looove the lectures! Thank you for the upload!
@lesliefontenelle7224 7 years ago

I am not involved in information technology but this lecturer is making a difficult subject like information theory look so easy. You really must see this...
@abhishekpal5871 8 years ago

this really helps. I really wanted to learn information theory. this video series is really easy to understand and awesome.
@the_anuragsrivastava 3 years ago

One of the best lecture of " information theory and coding " I have ever seen....love from India 🇮🇳🇮🇳🇮🇳🇮🇳
@linlinzhao9085 5 years ago

Dr. Mackay is a great explainer. Anyone interested in machine learning and Bayesian statistics can also read his doctoral thesis.
@qeithwreid7745 4 years ago

This is fantastic I love it.
@george5120 4 years ago

So nice to watch a video like this that does not have music.
@JerryFrenchJr 7 years ago

How am I just now discovering this lecture series??! This is awesome!