A very intuitive yet powerful inequality in information theory is the data processing inequality. Lemma: If random variable $latex X$, $latex Y$ and $latex Z$ form a Markov chain $latex X \rightarrow Y \rightarrow Z$, then $latex I(X;Y) \ge I(X;Z)$. The great thing about the inequality is that unlike some results in information theory, it…
Lemma 1: All eigenvalues of a Hermitian matrix are real. Proof: Let $latex A$ be Hermitian and $latex \lambda$ and $latex x$ be an eigenvalue and the corresponding eigenvector of $latex A$. We have $latex \lambda^* x^H x = (x^H A^H) x = x^H (A x)=\lambda x^Hx$. Thus we have $latex \lambda^*=\lambda$ as $latex x^H…
For a matrix $latex M = \begin{pmatrix} A &B\\C&D\end{pmatrix}$, we call $latex S\triangleq D -CA^{-1}B$ the Schur complement of $latex A$ in $latex M$. Note that $latex S$ naturally appear in block matrix inversion. Note that when $latex M$ is symmetric and $latex A$ is positive definite, $latex M$ is positive definite if and only…
Here are some formula for matrix inversion. Lemma 1: For a block matrix $latex M=\begin{pmatrix}A & B \\C &D\end{pmatrix}$, $latex M^{-1}=\begin{pmatrix}(A-B D^{-1} C)^{-1}& -A^{-1}B(D-CA^{-1}B)^{-1}\\-(D-CA^{-1}B)^{-1}CA^{-1}&(D-CA^{-1}B)^{-1}\end{pmatrix}$ $latex =\begin{pmatrix}A^{-1}+A^{-1}BS^{-1}CA^{-1}& -A^{-1}BS^{-1}\\-S^{-1}CA^{-1}&S^{-1}\end{pmatrix}$, where $latex S=D-CA^{-1}B$ is basically the Schur’s complement of block $latex A$. Proof: Let $latex M^{-1}=\begin{pmatrix}E&F\\G&H\end{pmatrix}$, $latex M M^{-1}=1$ gives us $latex AE+BG=I$ $latex AF+BH=0$ $latex CE+DG=0$ $latex…
After some trial and error and reading some other posts, I finally manage to capture screencast of a selected window with ffmpeg under 10.04. I am using the native ffmpeg from ubuntu and so recompiling is not needed. The command I’m using is as below. ffmpeg -f alsa -ac 2 -i pulse -f x11grab -r…
What can I say? I just can’t stop watching and and basically used up my whole day to finish the first season. Thinking back, it didn’t incorporate exactly lots of new elements. After all, the studio is Sunrise. It has been making war animes for more than 30 years. It just did everything they used…
Here is some quick note for the existence of eigenvector for linear operator $latex T$ in complex vector space of dimension $latex n$. Consider any non-zero vector $latex X$ in the space, we have the $latex n+1$ vectors $latex X, T[X],T^2[X],T^3[X],\cdots,T^n[X]$ to be linearly dependent. Thus, there exists $latex a_0,a_1,\cdots,a_n$ such that $latex a_0+a_1T[X]+a_2T^2[X]+\cdots+a_nT^n[X]=0$. If…
One thing I found it amazing is how average americans detest and even are terrified by the word socialism. Sometimes I heard people will say chineses are “communists” and “socialists”. It is interesting that America is actually a much more socialistic country than China. It is true that the chinese government is dictatorship. But it…
I finally finished Steins Gate. Honestly, I was a little bit disappointed. It is still good, even great. But I don’t think it is a classic as some may suggest. And the quality (both visual and story-wise) definitely declines about mid way until it came back up again near the end. I guess it is…