Dimensionality Reduction
No account yet ?
Dimensionality reduction makes it possible to project data from a high-dimensional space into a lower-dimensional space.
This operation is crucial in machine learning to fight against the scourge of large dimensions that alter the effectiveness of methods.
Dimensionality reduction is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension.
Working in high-dimensional spaces can be undesirable for many reasons because raw data is often sparse due to the curse of dimensionality, and data analysis is usually computationally intractable.
Dimensionality reduction is common in fields that deal with a large number of observations and a large number of variables, such as signal processing, speech recognition, neuroinformatics, and bioinformatics.
The methods are generally divided into linear and nonlinear approaches.
The approaches can also be divided into feature selection and feature extraction.
Dimensionality reduction can be used for noise reduction, data visualization, cluster analysis, or as an intermediate step to facilitate other analyses.
Machine learning algorithms rely on factors known as variables.
The higher the number of characteristics, the more difficult it is to visualize the training data and work on it.
Nevertheless, it happens that most of the characteristics are correlated which leads to redundancy.
It is to fight against the scourge of dimensionality that dimensionality reduction algorithms come into play.
There are several methods for dimensionality reduction.
Principal component analysis which is an unsupervised machine learning algorithm that can be used to visualize data that has more than three dimensions.
Linear discriminant analysis is a supervised machine learning algorithm that is used for dimensionality reduction, its approach is similar to that of PCA.
The LDA finds the components that maximize both the data variance and the separation between the multiple classes.
Dimensionality reduction can be linear or non-linear depending on the method used.
The most commonly used linear method is principal component analysis (PCA).
Dimensionality reduction is useful for problems that can arise from high dimensional data and can number in the millions.
This allows to keep only the discriminating characteristics to avoid overfitting of the models.
Python's Scikit-learn machine learning library implements the sklearn.decomposition and sklearn.discriminant_analysis modules to use dimensionality reduction.
The dataset used is that of iris.
Tested in Anaconda and Python 3.7
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
# import de l'ensemble de données
iris = datasets.load_iris()
X = iris.data
y = iris.target
cibles = iris.target_names
print(cibles)
df = pd.DataFrame(iris.data, columns = iris.feature_names)
df['Species'] = iris['target']
df['Species'] = df['Species'].apply(lambda x: iris['target_names'][x])
print(df.head())
couleur = {'Setosa' : 'blue', 'Versicolor' : 'red', 'Virginica' : 'orange'}
#Voyons comment les classes sont séparées en fonction des différentes caractéristiques
sns.FacetGrid(df, hue = "Species", height = 4,
palette = couleur.values()).map(plt.scatter, "sepal length (cm)",
"sepal width (cm)").add_legend()
sns.FacetGrid(df, hue = "Species", height = 4,
palette = couleur.values()).map(plt.scatter, "petal length (cm)",
"petal width (cm)").add_legend()
plt.show()
#creer une matrice de correlation
print(df.corr())
sns.heatmap(df.corr(), cmap = 'PiYG', annot = True)
#normalisation des données en utilisant standard scaler
scaler = StandardScaler()
X = scaler.fit_transform(X)
plt.show()
#PCA avec deux composantes
#L’analyse en composantes principales PCA
pca = PCA(n_components=2)
X_reduit = pca.fit_transform(X)
for color, i, cibles in zip(couleur.values(), [0, 1, 2], cibles):
plt.scatter(X_reduit[y == i, 0], X_reduit[y == i, 1], color = color, alpha = .8,
label = cibles, s = 130, edgecolors = 'k')
plt.legend(loc = 'best', shadow = False, scatterpoints = 1)
plt.xlabel("1ère composante du PCA")
plt.ylabel("2e composante du PCA")
plt.title('PCA de la dataset iris')
# pourcentage de la variance expliquée pour chaque composantes
print('variance expliquée pour chaque composantes: %s' % str(pca.explained_variance_ratio_))
plt.show()
#PCA avec trois composantes
from mpl_toolkits.mplot3d import Axes3D
figure = plt.figure(1, figsize = (8, 6))
axe = Axes3D(figure, elev =- 150, azim = 110)
pca3 = PCA(n_components = 3)
X_reduit = pca3.fit_transform(iris.data)
axe.scatter(X_reduit[:, 0], X_reduit[:, 1], X_reduit[:, 2], c = y, cmap = plt.cm.spring, edgecolor = 'k', s = 130)
axe.set_title("trois premieres composantes du PCA")
axe.set_xlabel("1ère composante du PCA")
axe.w_xaxis.set_ticklabels([])
axe.set_ylabel("2e composante du PCA")
axe.w_yaxis.set_ticklabels([])
axe.set_zlabel("3e composante du PCA")
axe.w_zaxis.set_ticklabels([])
# pourcentage de la variance expliquée pour chaque composantes
print('variance expliquée pour chaque composantes: {}'.format(pca3.explained_variance_ratio_))
plt.show()
#LDA avec deux composantes
couleur = {'Setosa' : 'blue','Versicolor' : 'red','Virginica' : 'orange'}
lda = LinearDiscriminantAnalysis(n_components=2)
lda.fit(X, y)
X_reduit = lda.transform(X)
plt.figure(figsize = (10,8))
for cl, i, cible in zip(couleur.values(), [0, 1, 2], cibles):
plt.scatter(X_reduit[y == i, 0], X_reduit[y == i, 1], alpha = .8, color = cl,
label = cible, s = 130, edgecolors = 'k')
plt.legend(loc = 3, shadow = False, scatterpoints = 1)
plt.xlabel('lda1')
plt.ylabel('lda2')
plt.title("Projection de l'iris sur les 2 premiers discriminants linéaires")
print('variance expliquée pour chaque composantes: {}'.format(lda.explained_variance_ratio_))
plt.show()
Source : https://www.cours-gratuit.com/tutoriel-python/tutoriel-python-les-algorithmes-de-rduction-de-dimensionnalit-avec-scikit-learn
PCA with two components
PCA with three components
LDA with two components
Welcome, my name is Eric Soupet and I am the administrator of the site elodees.com. elodees.com is a state of the art of Artificial Intelligence and aims to be collaborative, you can now offer content such as articles, events, tutorials, ... so don't hesitate !
