Skip to content

Linear Algebra & PCA Studio

Rotate the violet probe axis. Every point is projected onto it (orange drops); the variance of those projections is plotted against θ. Where it peaks is the first principal component: the eigenvector of the covariance matrix with the largest eigenvalue.

θ = 30°, variance along the probe 1.628

Cloud with the probe axis, projections and principal axes

PC1 λ = 1.72PC2 λ = 0.28
Cloud with the probe axis, projections and principal axes: θ = 30°, Var = 1.628; PC1 at 45.0° with λ₁ = 1.725, λ₂ = 0.275.

Variance of the projections vs θ

λ1 = 1.72λ2 = 0.280°90°180°
Variance of the projections vs θ: maximum λ₁ = 1.725 at 45.0°, minimum λ₂ = 0.275; now 1.628 at 30°.

Formula

Var(uᵀx) = uᵀΣu, ‖u‖ = 1max over u ⇒ Σu = λu, λ₁ + λ₂ = trace Σ

The projections onto PC1 and PC2 are uncorrelated: rotating to the eigenvectors is what 'removing correlation' means geometrically. PCA is a rotation; nothing latent is assumed.

Method: Cyclic Jacobi eigen-decomposition (src/lib/engine/linalg.ts); PCA on the covariance or correlation matrix of the data, loadings = eigenvector × √eigenvalue; principal-axis factoring from squared multiple correlations, at most 100 iterations, unrotated (rotation is not implemented); k-means is Lloyd's algorithm from seeded distinct starting points, empty clusters keep their centroid. Probe cloud and blobs are synthetic and seeded; item sets are ds.efficacy-120 and ds.index-120. · Synthetic, seeded data. Illustrative, not an empirical finding.

Validation: Eigenpairs match numpy.linalg.eigh (1e-12), PCA matches scikit-learn PCA on standardised items, one-factor PAF matches an independent numpy implementation (1e-9) and factor_analyzer MINRES (2e-4), and k-means reaches scikit-learn KMeans' inertia, labels and centres from the same start (tests/unit/pca-studio.test.ts). Identities checked: Var at PC1 = λ₁, Σλ = trace, SSW never increases. · numerically validated