Lie algebra is a vector space $latex g$ with a map $latex [\cdot, \cdot]:g\times g \rightarrow g$ such that $latex [\cdot,\cdot]$ is bilinear $latex [x,x]=0 $ Jacobi inequality: $latex [x,[y,z]]+[y,[z,x]]+[z,[x,y]]=0,\forall x,y,z$. Note that 2) $latex \Rightarrow [x,y]=-[y,x]$ since $latex 0=[x+y,x+y]=[x,x]+[y,y]+[x,y]+[y,x]$. Note that the converse is true most of time as well since that implies $latex…
The “S” in stands for special, meaning that , then . . The condition obviously requires . It turns out that the converse is true as well and so as shown in the following. Let , , and . Then any matrix in can be represented by . By the linearity of . We…
For the orthogonal set , we can define the Lie algebra Note that iff . First note that And since if , . Therefore . Now for the opposite direction, since , then Let . Remark: is topologically closed. That is, it contains its limit. Define as a mapping from any matrix to . Then…
DDPM claims to perform better than GANs in a very recent article. The idea of DDPM is that an image can progressively be added with noise resulting in a white noise image. And the neural networks can be trained to perform the reverse process, converting a white noise image to something looks natural.
Found a very nice series on Lie group and Lie algebra. Watched the first couple lectures. The main message I got so far is that Lie algebra combine techniques from analysis (calculus) with algebra (group). And one nice thing I learned is that $latex \exp(A+B) = \exp(A) \exp(B)$ if $latex AB = BA$. I didn’t…
Came across Weiszfeld’s algorithm while reading the quaternion equivariant capsule paper. Weiszfeld’s algorithm is just a form of iteratively reweighted least square. And the algorithm is well illustrated with this tweet.
Any square matrix can be decomposed into Jordan normal form. The Jordan block can be decomposed into different numbers of Jordan boxes depending on the geometric multiplicity of the corresponding eigenvalues (only one box if the corresponding geometric multiplicity is one). This video series gives an excellent description to compute the Jordan normal form.
A motivating example of hindsight experience replay is a bit flipping problem. Say start with a binary number as an initial state, an action can be flipping any arbitrary bit, and the goal is to reach a particular state. It is, of course, a toy problem that can be solved easily. However, say if we…
Example: Am -> x8989x -> E7
Didn’t realize that this piece is related to Paganini.