Cross validation

No account yet ?

Cross-validation is one of several similar model validation techniques for evaluating how the results of a statistical analysis will generalize to an independent data set.

Cross-validation is a resampling method that uses different parts of the data to test and train a model over different iterations.

It is mainly used in contexts where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice.

In a prediction problem, a model receives a data set of known data against which training is performed, and a data set of unknown data against which the model is tested.

The purpose of cross-validation is to test the ability of the model to predict new data that was not used to estimate it.

This is to flag issues such as overfitting or selection bias and to provide insight into how the model will generalize to an independent data set.

Machine Learning - Cross Validation K-Fold

Tested in Anaconda and Python 3.7

from sklearn import datasets
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import KFold, cross_val_score
 
X, y = datasets.load_iris(return_X_y=True)
 
clf = DecisionTreeClassifier(random_state=42)
 
k_folds = KFold(n_splits = 5)
 
scores = cross_val_score(clf, X, y, cv = k_folds)
 
print("Cross Validation Scores: ", scores)
print("Average CV Score: ", scores.mean())
print("Number of CV Scores used in Average: ", len(scores))

Source : https://www.w3schools.com/python/python_ml_cross_validation.asp

Cross Validation Scores: [1. 1. 0.83333333 0.93333333 0.8 ]

Average CV Score: 0.9133333333333333

Number of CV Scores used in Average: 5

Machine Learning - Cross Validation Stratified K-Fold

Tested in Anaconda and Python 3.7

from sklearn import datasets
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import StratifiedKFold, cross_val_score
 
X, y = datasets.load_iris(return_X_y=True)
 
clf = DecisionTreeClassifier(random_state=42)
 
sk_folds = StratifiedKFold(n_splits = 5)
 
scores = cross_val_score(clf, X, y, cv = sk_folds)
 
print("Cross Validation Scores: ", scores)
print("Average CV Score: ", scores.mean())
print("Number of CV Scores used in Average: ", len(scores))

Source : https://www.w3schools.com/python/python_ml_cross_validation.asp

Cross Validation Scores: [0.96666667 0.96666667 0.9 0.93333333 1. ]

Average CV Score: 0.9533333333333334

Number of CV Scores used in Average: 5

Machine Learning - Cross Validation Leave One Out (LOO)

Tested in Anaconda and Python 3.7

from sklearn import datasets
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import LeaveOneOut, cross_val_score
 
X, y = datasets.load_iris(return_X_y=True)
 
clf = DecisionTreeClassifier(random_state=42)
 
loo = LeaveOneOut()
 
scores = cross_val_score(clf, X, y, cv = loo)
 
print("Cross Validation Scores: ", scores)
print("Average CV Score: ", scores.mean())
print("Number of CV Scores used in Average: ", len(scores))

Source : https://www.w3schools.com/python/python_ml_cross_validation.asp

Cross Validation Scores :
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1.
1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.
1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 0. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1.]

Average CV Score: 0.94

Number of CV Scores used in Average: 150

Machine Learning - Cross Validation Leave P Out (LPO)

Tested in Anaconda and Python 3.7

from sklearn import datasets
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import LeavePOut, cross_val_score
 
X, y = datasets.load_iris(return_X_y=True)
 
clf = DecisionTreeClassifier(random_state=42)
 
lpo = LeavePOut(p=2)
 
scores = cross_val_score(clf, X, y, cv = lpo)
 
print("Cross Validation Scores: ", scores)
print("Average CV Score: ", scores.mean())
print("Number of CV Scores used in Average: ", len(scores))

Source : https://www.w3schools.com/python/python_ml_cross_validation.asp

Cross Validation Scores: [1. 1. 1. ... 1. 1. 1.]

Average CV Score: 0.9382997762863534

Number of CV Scores used in Average: 11175

Machine Learning - Cross Validation Shuffle Split

Tested in Anaconda and Python 3.7

from sklearn import datasets
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import ShuffleSplit, cross_val_score
 
X, y = datasets.load_iris(return_X_y=True)
 
clf = DecisionTreeClassifier(random_state=42)
 
ss = ShuffleSplit(train_size=0.6, test_size=0.3, n_splits = 5)
 
scores = cross_val_score(clf, X, y, cv = ss)
 
print("Cross Validation Scores: ", scores)
print("Average CV Score: ", scores.mean())
print("Number of CV Scores used in Average: ", len(scores))

Source : https://www.w3schools.com/python/python_ml_cross_validation.asp

Cross Validation Scores: [0.97777778 0.93333333 0.88888889 0.95555556 0.93333333]

Average CV Score: 0.9377777777777778

Number of CV Scores used in Average: 5

Deep learning

Machine learning

Welcome, my name is Eric Soupet and I am the administrator of the site elodees.com. elodees.com is a state of the art of Artificial Intelligence and aims to be collaborative, you can now offer content such as articles, events, tutorials, ... so don't hesitate !