I came across the proof of Perron Frobenius theorem here and it used Banach fixed point theorem. So I spent some time to understand the fixed point theorem from the wiki. Theorem Banach fixed point theorem considers a complete metric space (with metric ) and for any contraction mapping , there will exist a…
IMHO, the Levy process is just a generalization of the Poisson process. For example, consider $latex X_t$ as the number of arrivals by time $latex t$ as in the Poisson process. We expect the followings will be satisfied $latex X_t =0$ for $latex t = 0$ Random variables $latex X_{t_2} – X_{t_1}$ and $latex X_{t_4}-X_{t_3}$…
I think I knew how to show it before. But suddenly just realize I can’t. Say let’s define $latex F(w) = \frac{1}{2 \pi} \int f(t) e^{-i wt} dt$ and the inverse transform $latex \hat{f}(t)= \int F(w) e^{i w t}dw = f(t)$. So we expect $latex f(t)=\frac{1}{2 \pi} \int \int f(\tau) e^{-i w\tau} d\tau e^{iwt}dw$ $latex…
When we model probability of a variable $latex x$ by $latex p(x) = {e^{-\frac{F(x)}{T}}}$, $latex F(x)$ is often referred to as the free energy. The name is coming from historical reason. The Gibbs-Boltzmann distribution for a configuration is proportional to $latex e^{-\frac{H}{k_B T}}$. And the closest reason I found is from here $latex p(H,T) =…
I wasn’t aware that convex conjugate is just Legendre transformation. That is, $latex f^{*}(p)=\sup _{\tilde {x}}\{\langle p,{\tilde {x}}\rangle -f({\tilde {x}})\}\geq \langle p,x\rangle -f(x)$ Note that it is also known as the Legendre-Fenchel transformation. Btw, we have the Fenchel inequality $latex \langle p,x\rangle \le f(x)+f^*(p)$ directly from the definition since $latex f^{*}(p)=\sup _{\tilde {x}}\{\langle p,{\tilde…
For a random variable $latex X$, we can simply define a moment generating function as $latex MG(t) \triangleq E[e^{t X}]$. Then, $latex MG(t)^{(n)}|_{t=0} = E[X^{n}e^{tX}]|_{t=0}=E[X^n]$ is simply the $latex n$-th moment of $latex X$. Easy to verify that $latex \mathcal{N}(0,\sigma^2)$ has moment generating function of $latex e^{\frac{t^2\sigma^2}{2}}$ since $latex E[e^{tX}]=\frac{1}{\sqrt{2\pi\sigma^2}}\int e^{-\frac{x^2}{2\sigma^2}}e^{tx}dx=\frac{1}{\sqrt{2\pi\sigma^2}}\int e^{-\frac{1}{2\sigma^2}[(x-t\sigma^2)^2-t^2\sigma^4]}dx=e^{\frac{t^2\sigma^2}{2}}$ Central limit theorem…
As the name suggested, generalized linear model (GLM) is a generalization of the linear regression model. In a classic linear model, with input $latex X$, the output $latex Y$ is modeled by $latex Y|X \sim \mathcal{N} (B X, \sigma^2 I)$. GLM extends the model in two ways, first instead of having the mean as simple…
The set of all positive semi-definite matrices $latex \mathcal S^+$ forms a cone. Because for any two matrices $latex A$ and $latex B$ in $latex \mathcal S^+$. $latex \theta_1 A + \theta_2 B$ is still PSD and thus in $latex \mathcal S^+$. PSD is self-dual In the context of cone, $latex \mathcal S^+$ is self-dual….
I always forget what does pareto optimality mean. I reviewed the slides of convex optimization course by Stephen Boyd recently and found the following very intuitive example. The curve below shows that to maintain a certain production volume, a factory needs to allocate its resources (fuel and labor) in the region . Essentially, we can…
Always forgot the definition of this one. Wiki gave a very good explanation. It is just continuously differentiable function with additional constraint that the magnitudes of the derivatives cannot be too big (i.e., function can’t change too quickly).