Notes for Elements of Information Theory

• 1 Introduction and Preview
  • 1.1 Preview of the Book
• 2 Entropy, Relative Entropy, And Mutual Information
  • 2.1 Entropy
  • 2.2 Joint Entropy and Conditional Entropy
  • 2.3 Relative Entropy and Mutual Information
  • 2.4 Relationship Between Entropy and Mutual Information
  • 2.5 Chain Rules for Entropy, Relative Entropy, and Mutual Information

About

This is my notes for Elements of Information Theory by Thomas M. Cover and Joy A. Thomas.

Contributors

• Wufei “Mofii” Ma