IMHO, the Levy process is just a generalization of the Poisson process. For example, consider $latex X_t$ as the number of arrivals by time $latex t$ as in the Poisson process. We expect the followings will be satisfied $latex X_t =0$ for $latex t = 0$ Random variables $latex X_{t_2} – X_{t_1}$ and $latex X_{t_4}-X_{t_3}$…
I think I knew how to show it before. But suddenly just realize I can’t. Say let’s define $latex F(w) = \frac{1}{2 \pi} \int f(t) e^{-i wt} dt$ and the inverse transform $latex \hat{f}(t)= \int F(w) e^{i w t}dw = f(t)$. So we expect $latex f(t)=\frac{1}{2 \pi} \int \int f(\tau) e^{-i w\tau} d\tau e^{iwt}dw$ $latex…
By Shannon-Nyquist theorem, a bandlimited signal can be completely reconstructed with $latex 2W$ samples per second. Think of the opposite way, this means that each second we have $latex 2W$ independent components that we can vary for each second. The total noise power per second is $latex 2 W (N_0/2) = N_0 W$ and as…
I guess I seem to be confused with the partition function from evidence. When they appear together, I don’t seem to be confused. But when they appeared in different contexts, I just got confused like falling into some illusion tricks. Given data and parameter , the evidence is simply . And , where the denominator…
It sounds like a misnomer to me. I probably will just call it a “vector” representation. It doesn’t have the “distributed” meaning of scattering information into different places. For example, to recognize a cat with “distributed” representation, we may distribute features into like “does it has a tail?”, “does it have four legs?”, and “does…
LeCun has a new course on deep learning this spring. I found two things he mentioned that worth jotting down. First, natural data lives in low-dimensional manifold. Probably I should have came across that before but it didn’t register earlier. Come to think of it. This is a very important fact. Second, as it is…
Asian disease problem illustrated that framing can alter one’s decision based on if we are emphasizing gain or loss. Prospect theory is just a fancy name to conjecture what happens when the utility function is indeed what economists believe.
A good slide from a workshop.
When we model probability of a variable $latex x$ by $latex p(x) = {e^{-\frac{F(x)}{T}}}$, $latex F(x)$ is often referred to as the free energy. The name is coming from historical reason. The Gibbs-Boltzmann distribution for a configuration is proportional to $latex e^{-\frac{H}{k_B T}}$. And the closest reason I found is from here $latex p(H,T) =…
I wasn’t aware that convex conjugate is just Legendre transformation. That is, $latex f^{*}(p)=\sup _{\tilde {x}}\{\langle p,{\tilde {x}}\rangle -f({\tilde {x}})\}\geq \langle p,x\rangle -f(x)$ Note that it is also known as the Legendre-Fenchel transformation. Btw, we have the Fenchel inequality $latex \langle p,x\rangle \le f(x)+f^*(p)$ directly from the definition since $latex f^{*}(p)=\sup _{\tilde {x}}\{\langle p,{\tilde…