gen-briefly-explain-mutual-information-and-one-common-misconcept-1e82

Briefly explain mutual information and one common misconception about it.

## Mutual Information **Mutual information** measures how much knowing one random variable reduces uncertainty about another. Formally, it quantifies the reduction in entropy (uncertainty) of variable X when Y is observed, and vice versa. $$I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)$$ where H denotes entropy. It ranges from 0 (complete independence) to the minimum entropy of either variable (perfect dependence). ## Common Misconception **Misconception: Mutual information only captures linear relationships** Many assume mutual information is similar to correlation and only detects linear associations. In reality, mutual information captures **any** type of dependency—linear, nonlinear, monotonic, or complex. For example, if Y = X², mutual information will detect the relationship even though Pearson correlation might be near zero. This makes it more general than correlation, though also more computationally intensive to estimate.