Day: October 22, 2012

A very intuitive yet powerful inequality in information theory is the data processing inequality. Lemma: If random variable $latex X$, $latex Y$ and $latex Z$ form a Markov chain $latex X \rightarrow Y \rightarrow Z$, then $latex I(X;Y) \ge I(X;Z)$. The great thing about the inequality is that unlike some results in information theory, it…