gen-briefly-explain-conditional-entropy-and-one-common-misconcep-c3af

Briefly explain conditional entropy and one common misconception about it.

# Conditional Entropy **Definition:** Conditional entropy H(Y|X) measures the average amount of uncertainty remaining about random variable Y after observing random variable X. Mathematically: H(Y|X) = Σ p(x) H(Y|X=x) It quantifies how many bits, on average, are still needed to describe Y once X is known. ## Common Misconception **Misconception:** "Conditional entropy H(Y|X) equals the entropy of Y restricted to cases where X occurs." **Reality:** Conditional entropy is a *weighted average* over all possible values of X, not the entropy calculated on a subset of data. It's the expected value of the entropy of Y given each specific value of X, weighted by how often each X value occurs. For example, H(Y|X) ≠ H(Y∩X) or H(Y where X=true). Instead, it averages the uncertainty across all conditioning cases, which is why H(Y|X) ≤ H(Y) — conditioning never increases average uncertainty.