Residual Component Analysis: Generalising PCA for more flexible ...

Low-rank + Sparse-Inverse Covariance
(Kalaitzis & Lawrence, 12)
p(Λ) ∝ exp(−λ�Λ�1)
p(z|Λ) = N (0, Λ −1 )
p(x) = N (0, I)
p(y|x, z) = N (Wx + z, σ 2 I)
◮ Marginalising x and z from the joint log-density gives the
objective
log{p(Y|Λ)p(Λ)} =
n�
log{N (yi,:|0, WW ⊤ + ΣGlasso)p(Λ)}
i=1
�
(bounded below by) ≥
q(Z) log
p(Y, Z, Λ)
dZ
q(Z)
◮ Lower bound is maximised by a hybrid EM/RCA algorithm.