CodeBlanks Logo Icon CodeBlanks
Login Register

Stratified

Stratified k-fold preserves class distribution in each fold.

StratifiedKFold

In classification the folds must keep the class proportions; otherwise one fold might have no minority class at all. StratifiedKFold guarantees this:

from sklearn.model_selection import StratifiedKFold
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
for egitim, dogrulama in skf.split(X, y):   # y is needed
    print(np.bincount(y[dogrulama]))

This time split also takes y, because it keeps the proportions by looking at it.

Grouped and time-based splitting

If several measurements of the same patient are in both training and validation, the model "recognises" the patient and the score inflates. GroupKFold keeps all examples of a group in the same fold. For time series, TimeSeriesSplit always trains on the past and validates on the future:

from sklearn.model_selection import GroupKFold, TimeSeriesSplit
cross_val_score(model, X, y, cv=GroupKFold(5), groups=hasta_id)
for tr, va in TimeSeriesSplit(n_splits=3).split(X):
    print(tr.max() < va.min())   # True: training always comes first

The right splitter is the one that mimics how the data will really be used.

Practice Fold proportions

For y made of 12 zeros and 3 ones, compute the number of 1s in the folds of KFold(n_splits=3) and StratifiedKFold(n_splits=3) (both without shuffling) as a list with y[dogrulama].sum(). Print the two lists on separate lines.

Starter code

model.py
import numpy as np
from sklearn.model_selection import KFold, StratifiedKFold

X = np.arange(15).reshape(-1, 1)
y = np.array([0] * 12 + [1] * 3)

# buraya yaz

Sample solution

model.py
import numpy as np
from sklearn.model_selection import KFold, StratifiedKFold

X = np.arange(15).reshape(-1, 1)
y = np.array([0] * 12 + [1] * 3)

for bolucu in [KFold(n_splits=3), StratifiedKFold(n_splits=3)]:
    print([int(y[dogrulama].sum()) for _, dogrulama in bolucu.split(X, y)])

Expected output

[0, 0, 3]
[1, 1, 1]

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 Why does <code>StratifiedKFold.split</code> need the <code>y</code> parameter?

  • It doesn't
  • To keep the class proportions Correct answer
  • To train the model
  • To compute scores

Explanation: Distribution into folds is done for each class separately.

2 Keep the examples of the same group in the same fold.

Question 
cross_val_score(model, X, y, cv=GroupKFold(5), ____=hasta_id)
Answer 
cross_val_score(model, X, y, cv=GroupKFold(5), groups=hasta_id)

Explanation: groups gives which group each example belongs to.

3 Match the situations with the splitters.

Term Meaning
Imbalanced classes StratifiedKFold
Measurements of the same patient GroupKFold
Daily sales TimeSeriesSplit
Independent samples KFold

All lessons in Intermediate

1 scikit-learn basics

  1. What is scikit-learn?
  2. Estimator API
  3. The fit method
  4. The predict method
  5. The score method
  6. Summary - scikit-learn

2 Pipeline

  1. What is a Pipeline?
  2. Steps
  3. fit and predict
  4. Advantages
  5. ColumnTransformer
  6. Summary - Pipeline

3 Cross-validation

  1. What is cross-validation?
  2. K-fold
  3. cross_val_score
  4. Stratified
  5. Advantages
  6. Summary - Cross-validation

4 Grid search

  1. What is grid search?
  2. GridSearchCV
  3. param_grid
  4. best_params_
  5. best_score_
  6. Summary - Grid search

5 SVM

  1. What is SVM?
  2. Margin
  3. Kernel
  4. C parameter
  5. Classification
  6. Summary - SVM

6 Project: SVM

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

7 Random forest

  1. What is random forest?
  2. Ensemble
  3. n_estimators
  4. Bagging
  5. Feature importance
  6. Summary - Random forest

8 Boosting

  1. What is boosting?
  2. AdaBoost
  3. Gradient boosting
  4. Learning rate
  5. Advantages
  6. Summary - Boosting

9 Clustering

  1. What is clustering?
  2. KMeans
  3. Centroid
  4. Elbow method
  5. Labels
  6. Summary - Clustering

10 KMeans in depth

  1. KMeans algorithm
  2. Initialization
  3. n_init
  4. inertia_
  5. random_state
  6. Summary - KMeans

11 Hierarchical clustering

  1. What is hierarchical clustering?
  2. Dendrogram
  3. Agglomerative
  4. Linkage
  5. Advantages
  6. Summary - Hierarchical clustering

12 Project: Clustering

  1. Preparing data
  2. Training KMeans
  3. Label analysis
  4. Printing results

13 PCA

  1. What is PCA?
  2. Variance
  3. n_components
  4. fit_transform
  5. explained_variance_
  6. Summary - PCA

14 Dimensionality reduction

  1. What is dimensionality reduction?
  2. Why reduce?
  3. PCA vs t-SNE
  4. Visualization
  5. Advantages
  6. Summary - Dimensionality reduction

15 Feature selection

  1. What is feature selection?
  2. Variance threshold
  3. SelectKBest
  4. Recursive elimination
  5. Feature importance
  6. Summary - Feature selection

16 Imbalanced data

  1. What is imbalanced data?
  2. SMOTE
  3. Class weight
  4. Oversampling
  5. Metric selection
  6. Summary - Imbalanced data

17 Project: Features

  1. Feature extraction
  2. Feature selection
  3. Training model
  4. Evaluation

18 Ensemble methods

  1. What is ensemble?
  2. Voting
  3. Bagging
  4. Boosting
  5. Stacking
  6. Summary - Ensemble

19 Regularization

  1. What is regularization?
  2. L1 (Lasso)
  3. L2 (Ridge)
  4. alpha
  5. ElasticNet
  6. Summary - Regularization

20 Intermediate final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4