Basic ideas of the Principal Component Analysis (PCA)
We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see below for its definition)
- Each data point is determined by \( p \) extrinsic (measurement) variables
- We may want to ask the following question: Are there fewer intrinsic variables (say \( d < < p \)) that still approximately describe the data?
- If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do.