The set of all positive semi-definite matrices $latex \mathcal S^+$ forms a cone. Because for any two matrices $latex A$ and $latex B$ in $latex \mathcal S^+$. $latex \theta_1 A + \theta_2 B$ is still PSD and thus in $latex \mathcal S^+$.
PSD is self-dual
In the context of cone, $latex \mathcal S^+$ is self-dual. The dual of $latex \mathcal S^+$ is $latex \mathcal D = \{M \in S| \langle A, B \rangle \ge 0, \forall B \in \mathcal S^+\}$. I was a bit confused earlier as I can see that this condition may require $latex \mathcal D$ to be “PSD”, but I don’t see how this condition can force $latex \mathcal D$ to be symmetric. It turns out that the symmetry is required by the definition directly (only restricted to symmetric matrices).
Now, for the PSD condition, consider any matrix $latex A \in \mathcal D$, and let $latex B = X X^\top$ and note that $latex B \in \mathcal S^+$. Thus, $latex \langle A, B \rangle = tr(A^\top B) = tr(A^\top X X^\top) = tr(X^\top A X)=X^\top A X \ge 0$. Since $latex X$ is arbitrary, $latex A$ is PSD.