gen-define-the-noisy-channel-theorem-and-give-one-short-worked-e-50af

Define the noisy-channel theorem and give one short worked example.

# Noisy-Channel Theorem **Shannon's Noisy-Channel Coding Theorem** states that for any communication channel with capacity C and any transmission rate R < C, there exists a coding scheme that allows data to be transmitted with arbitrarily low probability of error. Conversely, if R > C, the error probability cannot be made arbitrarily small. The channel capacity is given by: **C = max p(x) I(X;Y)** where I(X;Y) is the mutual information between input X and output Y. ## Worked Example: Binary Symmetric Channel (BSC) **Setup:** A BSC flips each bit with probability p = 0.1 (crossover probability). **Finding Capacity:** For a BSC: **C = 1 - H(p)** where H(p) = -p log₂(p) - (1-p) log₂(1-p) is the binary entropy function. **Calculation:** - H(0.1) = -0.1 log₂(0.1) - 0.9 log₂(0.9) - H(0.1) = -0.1(-3.32) - 0.9(-0.152) - H(0.1) ≈ 0.332 + 0.137 = 0.469 **C = 1 - 0.469 = 0.531 bits per transmission** **Interpretation:** We can reliably transmit at any rate up to 0.531 bits per channel use. For example, we could send 531 bits of information using 1000 channel uses with arbitrarily low error probability using appropriate error-correcting codes (like LDPC or Turbo codes).