# Principal Component Analysis >[!Abstract] >- What is principal component analysis? >- How does it work? >- Explain the sort of problems you would use PCA for >- Describe PCA's formulation and derivation in matrix form >- What are the pros and cons of PCA? Explain its limitations as a method ## What is PCA? PCA is a **[[Dimensionality Reduction 1|dimensionality reduction]]** technique that transforms input features into their principal components. It converts a set of observations of possibly correlated features into a set of values of **linearly uncorrelated** features. In other words, PCA aims to project data of $p$ dimensions onto a linear subspace $q\leq p$ dimensions capturing as much variance as possible in projected space. - Goal: map the data from the original high-dimensional space to a lower dimensional space that captures as much of the **variation** in the data as possible. It aims to find the most useful subset of dimensions to summarize the data. - Eg. a dataset with 3 features --> use PCA to extract the first principal component - **Linear transformation:** PCA finds a sequence of linear combinations of features that have maximum variance and are uncorrelated - PCA is an **unsupervised** learning method ![[pca_demo.png]] ![[lower_dimensional_pca.png]] ## How does PCA Work? ### General Idea Principal components are the directions of **maximum variance**, which has the effect of minimizing the information loss when you perform a projection or a compression down onto these principal components. >[!Question] Why does maximum variance lead to minimum information loss? > Support $x^{(i)}$ is an example in the original dataset, $v$ is the principle component (a vector) and $a^{(i)}v$ is the transformed data using PCA. ```Python from sklearn.decomposition import PCA import numpy as np # Generate some data np.random.seed(0) data = np.random.randn(100, 5) # Perform PCA pca = PCA(n_components=2) pca.fit(data) # Transform the data data_pca = pca.transform(data) # Print the explained variance ratio print(pca.explained_variance_ratio_) ``` ## Applications of PCA - Neuroscience - PCA is also used to find the **identity of a neuron** from the shape of its action potential. - PCA as a dimension reduction technique is used detect coordinated activities of large neuronal ensembles. It has been used in determining collective variables, that is, order parameters, during phase transitions in the brain. - Quantitative Finance -  Say, a fund manager has 200 stocks in his portfolio. To analyze these stocks quantiatively a stock manager will require a co-relational matrix of the size 200 * 200, which makes the problem very complex. However if he was to extract, 10 Principal Components which best represent the variance in the stocks best, this would reduce the complexity of problem while still explaining the movement of all 200 stocks. - Impage Compression ![[pca_compression.png]] - Facial Recognition - PCA could be used on a collection of face images to form a set of basis features. - Other Applications - PCA has been used in the Detection and Visualization of Computer Network Attacks. - PCA has been used in Anomaly Detection. ## Relations - [[Factor Analysis]] ## Sources - [Applications of Principal Component Analysis (PCA)](https://iq.opengenus.org/applications-of-pca/) - [Principal Component Analysis (PCA) | Machine Learning Interview Q&A for Data Scientists](https://www.youtube.com/watch?v=xswoz-6FXHE) - [Principal Component Analysis (PCA)](https://ml-notes-rajatgupta.notion.site/Principal-Component-Analysis-PCA-1f68e929a91c4215a43724df45febf5b)