CodeBlanks Logo Icon CodeBlanks
Login Register

Summary - Hierarchical clustering

In this group you learned hierarchical clustering and dendrograms.

Summary: hierarchical clustering

In this group:

  • Agglomerative clustering starts from every example and merges the closest clusters; the result is a tree.
  • The linkage matrix and the dendrogram; a large distance jump is a good cut point.
  • fcluster (maxclust/distance) and AgglomerativeClustering (n_clusters/distance_threshold).
  • Linkage types: single (chains), complete, average, ward (default).
  • It is deterministic; slow on large data; for new examples a classifier is trained on the labels.

Practice Dendrogram and cut

Compute the Ward linkage on the small data, draw the dendrogram and save it. Cut at the middle of the largest jump between consecutive merge distances ((Z[i, 2] + Z[i + 1, 2]) / 2, i = np.diff(Z[:, 2]).argmax()) and print the labels with fcluster(..., criterion="distance").

Starter code

model.py
import numpy as np
from scipy.cluster.hierarchy import linkage, fcluster

X = np.array([[0, 0], [0, 1], [1, 0], [5, 5], [5, 6], [6, 5], [12, 0], [12, 1]])
from scipy.cluster.hierarchy import dendrogram
import matplotlib.pyplot as plt

# buraya yaz

Sample solution

model.py
import numpy as np
from scipy.cluster.hierarchy import linkage, fcluster

X = np.array([[0, 0], [0, 1], [1, 0], [5, 5], [5, 6], [6, 5], [12, 0], [12, 1]])
from scipy.cluster.hierarchy import dendrogram
import matplotlib.pyplot as plt

Z = linkage(X, method="ward")
fig, ax = plt.subplots()
dendrogram(Z, ax=ax)
fig.savefig("agac.png")
i = np.diff(Z[:, 2]).argmax()
esik = (Z[i, 2] + Z[i + 1, 2]) / 2
print(fcluster(Z, t=esik, criterion="distance"))

Expected output

[2 2 2 3 3 3 1 1]

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 Which linkage produces clusters most similar to KMeans?

  • single
  • average
  • complete
  • ward Correct answer

Explanation: Like KMeans, Ward picks the merge that least increases the within-cluster squared distance.

2 Draw the dendrogram.

Question 
____(Z, ax=ax)
Answer 
dendrogram(Z, ax=ax)

Explanation: dendrogram lives in scipy.cluster.hierarchy and draws with matplotlib.

3 Match the tools.

Term Meaning
linkage Merge matrix
dendrogram Tree drawing
fcluster Cutting the tree
AgglomerativeClustering scikit-learn version

All lessons in Intermediate

1 scikit-learn basics

  1. What is scikit-learn?
  2. Estimator API
  3. The fit method
  4. The predict method
  5. The score method
  6. Summary - scikit-learn

2 Pipeline

  1. What is a Pipeline?
  2. Steps
  3. fit and predict
  4. Advantages
  5. ColumnTransformer
  6. Summary - Pipeline

3 Cross-validation

  1. What is cross-validation?
  2. K-fold
  3. cross_val_score
  4. Stratified
  5. Advantages
  6. Summary - Cross-validation

4 Grid search

  1. What is grid search?
  2. GridSearchCV
  3. param_grid
  4. best_params_
  5. best_score_
  6. Summary - Grid search

5 SVM

  1. What is SVM?
  2. Margin
  3. Kernel
  4. C parameter
  5. Classification
  6. Summary - SVM

6 Project: SVM

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

7 Random forest

  1. What is random forest?
  2. Ensemble
  3. n_estimators
  4. Bagging
  5. Feature importance
  6. Summary - Random forest

8 Boosting

  1. What is boosting?
  2. AdaBoost
  3. Gradient boosting
  4. Learning rate
  5. Advantages
  6. Summary - Boosting

9 Clustering

  1. What is clustering?
  2. KMeans
  3. Centroid
  4. Elbow method
  5. Labels
  6. Summary - Clustering

10 KMeans in depth

  1. KMeans algorithm
  2. Initialization
  3. n_init
  4. inertia_
  5. random_state
  6. Summary - KMeans

11 Hierarchical clustering

  1. What is hierarchical clustering?
  2. Dendrogram
  3. Agglomerative
  4. Linkage
  5. Advantages
  6. Summary - Hierarchical clustering

12 Project: Clustering

  1. Preparing data
  2. Training KMeans
  3. Label analysis
  4. Printing results

13 PCA

  1. What is PCA?
  2. Variance
  3. n_components
  4. fit_transform
  5. explained_variance_
  6. Summary - PCA

14 Dimensionality reduction

  1. What is dimensionality reduction?
  2. Why reduce?
  3. PCA vs t-SNE
  4. Visualization
  5. Advantages
  6. Summary - Dimensionality reduction

15 Feature selection

  1. What is feature selection?
  2. Variance threshold
  3. SelectKBest
  4. Recursive elimination
  5. Feature importance
  6. Summary - Feature selection

16 Imbalanced data

  1. What is imbalanced data?
  2. SMOTE
  3. Class weight
  4. Oversampling
  5. Metric selection
  6. Summary - Imbalanced data

17 Project: Features

  1. Feature extraction
  2. Feature selection
  3. Training model
  4. Evaluation

18 Ensemble methods

  1. What is ensemble?
  2. Voting
  3. Bagging
  4. Boosting
  5. Stacking
  6. Summary - Ensemble

19 Regularization

  1. What is regularization?
  2. L1 (Lasso)
  3. L2 (Ridge)
  4. alpha
  5. ElasticNet
  6. Summary - Regularization

20 Intermediate final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4