Elements of Information Theory

Written by Thomas M. Cover and Joy A. Thomas, the textbook teaches information theory through a mix of mathematics, physics and statistics. It covers the core subjects in detail, among them entropy and data compression, channel capacity, rate distortion, hypothesis testing and network information theory, with further chapters on gambling, Kolmogorov complexity and the stock market. Each chapter ends with problem sets, a compact summary and historical notes. The second edition reorganized the chapters for teaching, added 200 new problems, and brought in new material on feedback capacity, portfolio theory and source coding. Many regard it as a benchmark text on the subject.

Eric Weinstein, managing director of Thiel Capital, shared it on Twitter as part of four shelves of books he chose to keep after clearing out an office.