Mahalanobis Distance

Let $\mathbf{x}$ be a vector, and assume we have a distribution with mean $\mu$ and covariance matrix $S$. The Mahalanobis distance with respect to the distribution is defined as

$$D(\mathbf{x})=\sqrt{(\mathbf{x}–\mu {)}^T{S}^{-1}(\mathbf{x}–\mu )}.$$

Why? Well, $S$ is positive-definite, so we can decompose $\mathbf{x}–\mu$ into $\sum a_i{\mathbf{e}}_i$, where ${\mathbf{e}}_i$ are the orthonormal eigenvectors of $S$ (with corresponding eigenvalues ${\lambda }_i$). Then $S^{-1}{a}_i{\mathbf{e}}_i=\frac{a_i{\mathbf{e}}_i}{{\lambda }_i}$ and so

$$D(\mathbf{x})=\sqrt{\sum \frac{a_i^2}{{\lambda }_i}}.$$

So the Mahalanobis distance takes into account how far out from the mean we are, where the meaning of “far out” depends on the direction you’ve traveled from the mean.