Difference Between Agglomerative clustering and Divisive clustering

Hierarchical clustering is a popular unsupervised machine learning technique used to group similar data points into clusters based on their similarity or dissimilarity. It is called “hierarchical” because it creates a tree-like structure of clusters known as a dendrogram where each node represents a cluster that can be divided into smaller sub-clusters

There are two types of hierarchical clustering techniques: 

1. Agglomerative (Bottom-up approach)
2. Divisive clustering (Top-down approach)

Understanding Agglomerative Clustering

Agglomerative clustering is a bottom-up approach where each data point starts as its own individual cluster. The algorithm iteratively merges the most similar pairs of clusters until all the data points belong to a single cluster. It’s widely used due to its simplicity and efficiency in many clustering tasks.

Key steps in agglomerative clustering:

1. Treat each data point as a separate cluster.
2. Calculate the similarity (distance) between all pairs of clusters.
3. Merge the two most similar clusters.
4. Repeat steps 2-3 until all points belong to a single cluster.

This method can be computationally expensive especially for large datasets. The algorithm needs to compute the distance between every pair of points leading to a time complexity of $O(n^3)$ for large datasets.

Example:

It can be implemented using Scikit learn and SciPy library of python.

Here’s a simple implementation of agglomerative clustering using randomly generated data in Python with SciPy:

import numpy as np
import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage

data = np.random.randn(50, 2)

Z = linkage(data, 'ward')

# Plot dendrogram
plt.figure(figsize=(10, 7))
dendrogram(Z)
plt.title("Agglomerative Clustering Dendrogram")
plt.show()

Output:

Understanding Divisive Clustering

Divisive clustering on the other hand, is a top-down approach. It starts with all data points in a single cluster and recursively splits the clusters into smaller sub-clusters based on their dissimilarity until each data point is in its own individual cluster. This approach is more computationally intensive, as it requires splitting the data rather than merging it.

Key steps in divisive clustering:

1. Start with a single cluster containing all the data points.
2. Split the cluster into two sub-clusters based on their dissimilarity.
3. Recursively apply the same process to the resulting sub-clusters.
4. Repeat until each data point is in its own cluster.

Divisive clustering’s complexity can vary depending on the implementation it generally requires more computational power due to the recursive splitting process. However because it operates on sub-clusters it can sometimes reduce the computational cost when compared to agglomerative clustering on very large datasets

Divisive clustering is more complex to implement and requires a careful choice of splitting criteria making it less commonly used than agglomerative clustering that’s why it is not available in SciPy and Scikit learn.

Difference between agglomerative clustering and Divisive clustering :

Both agglomerative and divisive clustering are powerful hierarchical clustering techniques with their own strengths and weaknesses. Agglomerative clustering is more commonly used due to its simplicity and efficiency while divisive clustering may be useful in specific applications where a top-down approach is preferred. Understanding these methods and their differences will help in selecting the appropriate technique for a given clustering task.

harshitmongre

Follow

Improve

Article Tags :

• AI-ML-DS
• Machine Learning
• ML-Clustering
• python

Practice Tags :

• Machine Learning
• python