CodeBlanks Logo Icon CodeBlanks
Login Register

Agglomerative

Agglomerative starts each example as a cluster and merges them.

AgglomerativeClustering

Its scikit-learn counterpart is AgglomerativeClustering. Either the number of clusters or a distance threshold is given:

from sklearn.cluster import AgglomerativeClustering
ag = AgglomerativeClustering(n_clusters=3, linkage="ward")
etiket = ag.fit_predict(X)
ag2 = AgglomerativeClustering(n_clusters=None, distance_threshold=10).fit(X)
print(ag2.n_clusters_)   # cluster count from the threshold

Unlike KMeans there is no predict method: a new example can't be added to the tree; only the training data is labelled.

Scale and cost

Agglomerative clustering needs all pairwise distances; memory and time grow with the square of the number of examples. It slows down on tens of thousands of examples. One fix is allowing merges only between neighbours with connectivity:

from sklearn.neighbors import kneighbors_graph
bag = kneighbors_graph(X, n_neighbors=10, include_self=False)
ag = AgglomerativeClustering(n_clusters=3, connectivity=bag).fit(X)

A neighbourhood graph also helps find long, winding clusters.

Practice Cluster with a threshold

Train an AgglomerativeClustering(n_clusters=None, distance_threshold=30) on the synthetic data; print the resulting cluster count (n_clusters_). Then train with n_clusters=3 and print the ARI against the true labels with round(..., 3).

Starter code

model.py
from sklearn.datasets import make_blobs
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import adjusted_rand_score

X, y = make_blobs(n_samples=300, centers=3, cluster_std=[0.5, 1.5, 0.8], random_state=4)

# buraya yaz

Sample solution

model.py
from sklearn.datasets import make_blobs
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import adjusted_rand_score

X, y = make_blobs(n_samples=300, centers=3, cluster_std=[0.5, 1.5, 0.8], random_state=4)

ag = AgglomerativeClustering(n_clusters=None, distance_threshold=30).fit(X)
print(ag.n_clusters_)
etiket = AgglomerativeClustering(n_clusters=3).fit_predict(X)
print(round(adjusted_rand_score(y, etiket), 3))

Expected output

3
0.932

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 What is the way to cluster a new example with AgglomerativeClustering?

  • Using transform
  • Calling fcluster
  • There is none directly; retrain, or train a classifier on the labels Correct answer
  • Using predict

Explanation: The practical fix: train a KNN on the cluster labels and assign new examples with it.

2 Give a distance threshold instead of a cluster count.

Question 
ag = AgglomerativeClustering(n_clusters=None, ____=10)
Answer 
ag = AgglomerativeClustering(n_clusters=None, distance_threshold=10)

Explanation: When distance_threshold is given, n_clusters must be None.

3 Match the properties.

Term Meaning
n_clusters Fixed cluster count
distance_threshold Cut by threshold
n_clusters_ Resulting cluster count
connectivity Limiting by neighbourhood

All lessons in Intermediate

1 scikit-learn basics

  1. What is scikit-learn?
  2. Estimator API
  3. The fit method
  4. The predict method
  5. The score method
  6. Summary - scikit-learn

2 Pipeline

  1. What is a Pipeline?
  2. Steps
  3. fit and predict
  4. Advantages
  5. ColumnTransformer
  6. Summary - Pipeline

3 Cross-validation

  1. What is cross-validation?
  2. K-fold
  3. cross_val_score
  4. Stratified
  5. Advantages
  6. Summary - Cross-validation

4 Grid search

  1. What is grid search?
  2. GridSearchCV
  3. param_grid
  4. best_params_
  5. best_score_
  6. Summary - Grid search

5 SVM

  1. What is SVM?
  2. Margin
  3. Kernel
  4. C parameter
  5. Classification
  6. Summary - SVM

6 Project: SVM

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

7 Random forest

  1. What is random forest?
  2. Ensemble
  3. n_estimators
  4. Bagging
  5. Feature importance
  6. Summary - Random forest

8 Boosting

  1. What is boosting?
  2. AdaBoost
  3. Gradient boosting
  4. Learning rate
  5. Advantages
  6. Summary - Boosting

9 Clustering

  1. What is clustering?
  2. KMeans
  3. Centroid
  4. Elbow method
  5. Labels
  6. Summary - Clustering

10 KMeans in depth

  1. KMeans algorithm
  2. Initialization
  3. n_init
  4. inertia_
  5. random_state
  6. Summary - KMeans

11 Hierarchical clustering

  1. What is hierarchical clustering?
  2. Dendrogram
  3. Agglomerative
  4. Linkage
  5. Advantages
  6. Summary - Hierarchical clustering

12 Project: Clustering

  1. Preparing data
  2. Training KMeans
  3. Label analysis
  4. Printing results

13 PCA

  1. What is PCA?
  2. Variance
  3. n_components
  4. fit_transform
  5. explained_variance_
  6. Summary - PCA

14 Dimensionality reduction

  1. What is dimensionality reduction?
  2. Why reduce?
  3. PCA vs t-SNE
  4. Visualization
  5. Advantages
  6. Summary - Dimensionality reduction

15 Feature selection

  1. What is feature selection?
  2. Variance threshold
  3. SelectKBest
  4. Recursive elimination
  5. Feature importance
  6. Summary - Feature selection

16 Imbalanced data

  1. What is imbalanced data?
  2. SMOTE
  3. Class weight
  4. Oversampling
  5. Metric selection
  6. Summary - Imbalanced data

17 Project: Features

  1. Feature extraction
  2. Feature selection
  3. Training model
  4. Evaluation

18 Ensemble methods

  1. What is ensemble?
  2. Voting
  3. Bagging
  4. Boosting
  5. Stacking
  6. Summary - Ensemble

19 Regularization

  1. What is regularization?
  2. L1 (Lasso)
  3. L2 (Ridge)
  4. alpha
  5. ElasticNet
  6. Summary - Regularization

20 Intermediate final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4