Applications of Information Theory in Machine Learning
A detailed guide to applications of information theory in machine learning. Covers key methods, mathematical significance, and real-world applications.
Mathematics Category
Information Theory
21 articles
≈ 210 min total reading
Arithmetic Coding: Near-Optimal Compression
A detailed guide to arithmetic coding: near-optimal compression. Covers key methods, mathematical significance, and real-world applications.
Channel Capacity: Maximum Reliable Transmission Rate
A detailed guide to channel capacity: maximum reliable transmission rate. Covers key methods, mathematical significance, and real-world applications.
Convolutional Codes and Turbo Codes
A detailed guide to convolutional codes and turbo codes. Covers key methods, mathematical significance, and real-world applications.
Differential Entropy: Continuous Information
A detailed guide to differential entropy: continuous information. Covers key methods, mathematical significance, and real-world applications.
Entropy: Measuring Information and Uncertainty
A detailed guide to entropy: measuring information and uncertainty. Covers key methods, mathematical significance, and real-world applications.
Error-Correcting Codes: Hamming and Reed-Solomon
A detailed guide to error-correcting codes: hamming and reed-solomon. Covers key methods, mathematical significance, and real-world applications.
Gaussian Channel: Capacity and Water-Filling
A detailed guide to gaussian channel: capacity and water-filling. Covers key methods, mathematical significance, and real-world applications.
Huffman Coding: Optimal Prefix Codes
A detailed guide to huffman coding: optimal prefix codes. Covers key methods, mathematical significance, and real-world applications.
Information Theory
A detailed guide to information theory. Covers key methods, mathematical significance, and real-world applications.
Information Theory and Statistics: Fisher Information
A detailed guide to information theory and statistics: fisher information. Covers key methods, mathematical significance, and real-world applications.
Joint Entropy and Conditional Entropy
A detailed guide to joint entropy and conditional entropy. Covers key methods, mathematical significance, and real-world applications.
Kolmogorov Complexity: Algorithmic Information Theory
A detailed guide to kolmogorov complexity: algorithmic information theory. Covers key methods, mathematical significance, and real-world applications.
Kullback-Leibler Divergence: Measuring Distribution Differences
A detailed guide to kullback-leibler divergence: measuring distribution differences. Covers key methods, mathematical significance, and real-world applicat
Lempel-Ziv Compression: Dictionary-Based Methods
A detailed guide to lempel-ziv compression: dictionary-based methods. Covers key methods, mathematical significance, and real-world applications.
Mutual Information: Dependence Between Variables
A detailed guide to mutual information: dependence between variables. Covers key methods, mathematical significance, and real-world applications.
Network Information Theory: Multiple Access and Broadcast Channels
A detailed guide to network information theory: multiple access and broadcast channels. Covers key methods, mathematical significance, and real-world appli
Noisy Channel Coding Theorem: Shannon's Landmark Result
A detailed guide to noisy channel coding theorem: shannon's landmark result. Covers key methods, mathematical significance, and real-world applications.
Quantum Information Theory: Qubits and Entanglement
A detailed guide to quantum information theory: qubits and entanglement. Covers key methods, mathematical significance, and real-world applications.
Rate-Distortion Theory: Lossy Compression Limits
A detailed guide to rate-distortion theory: lossy compression limits. Covers key methods, mathematical significance, and real-world applications.
Source Coding Theorem: Lossless Compression Limits
A detailed guide to source coding theorem: lossless compression limits. Covers key methods, mathematical significance, and real-world applications.