Pseudo inverse of the covariance matrix?

I've been looking for methods to compute a pseudo inverse of a covariance matrix. And found that one way is to construct a regularized inverse matrix. By constructing the eigen system, and removing the least significant eigenvalues and then use the eigen values and vectors to form an approximate inverse.

Could anyone explain the idea behind this?

Thanks in advance