Linear Algebra & PCA Studio
Rotate the violet probe axis. Every point is projected onto it (orange drops); the variance of those projections is plotted against θ. Where it peaks is the first principal component: the eigenvector of the covariance matrix with the largest eigenvalue.
θ = 30°, variance along the probe 1.628
Cloud with the probe axis, projections and principal axes
Variance of the projections vs θ
Formula
Var(uᵀx) = uᵀΣu, ‖u‖ = 1max over u ⇒ Σu = λu, λ₁ + λ₂ = trace Σ
The projections onto PC1 and PC2 are uncorrelated: rotating to the eigenvectors is what 'removing correlation' means geometrically. PCA is a rotation; nothing latent is assumed.