Like Article. Previous K means Clustering - Introduction. Next Different Types of Clustering Algorithm. Recommended Articles. Article Contributed By :. Surya Priy. Easy Normal Medium Hard Expert. Writing code in comment? Please use ide. It is a partitioning method dividing the data space into K distinct clusters. It starts out with randomly-selected K cluster centers Figure 4, left , and all data points are assigned to the nearest cluster centers Figure 4, right.
Then the cluster centers are re-calculated as the centroids of the newly formed clusters. The data points are re-assigned to the nearest cluster centers we just re-calculated. This process, assigning data points to the cluster centers and re-calculating the cluster centers, is repeated until the cluster centers stop moving Figure 5. Clusters formed by k-Means clustering tend to be similar in sizes.
Moreover, clusters are convex-shaped. Also clustering results may be highly influenced by the choice of the initial cluster centers. Hierarchical clustering algorithm works by iteratively connecting closest data points to form clusters.
Initially all data points are disconnected from each other; each data point is treated as its own cluster. Then, the two closest data points are connected, forming a cluster. Next, the two next closest data points or clusters are connected to form a larger cluster. And so on. The process is repeated to form progressively larger clusters, and continues until all data points are connected into a single cluster Figure 6.
Hierarchical clustering forms a hierarchy of clusters, described in a diagram known as a dendrogram Figure 6, left. To obtain a cluster partition with a particular number of clusters, one can simply apply a cut-off threshold at a particular distance on the dendrogram, producing the desired number of clusters Figure 7. The shape of clusters, formed by hierarchical clustering, depends on how the distance is calculated between clusters.
In the single linkage method, the inter-cluster distance is measured by the closest two points between the two clusters Figure 8, left. On the other hand, in the complete linkage method, the distance is calculated as the farthest points between the two clusters Figure 9, left. In the average linkage method, the inter-cluster distance is calculated as the distance between the centers of gravity between two clusters.
This approach is a compromise between the single and complete linkage methods. It is a density-based clustering method, grouping dense clouds of data points into clusters. Any isolated points are considered not part of clusters, and are treated as noises. If there are a sufficiently large number of points within the neighborhood around that point, then those points are considered as part of the same cluster as the starting point.
These methods are referred to as linkage methods and will impact the results of the hierarchical clustering algorithm. Some of the most popular linkage methods include:. Complete linkage , which uses the maximum distance between any two points in each cluster. Single linkage , which uses the minimum distance between any two points in each cluster. Average linkage , which uses the average of the distance between each point in each cluster.
Euclidean distance is almost always the metric used to measure distance in clustering applications, as it represents distance in the physical world and is straightforward to understand, given that it comes from the Pythagorean theorem. You can again use an elbow plot to compare the within-cluster variation at each number of clusters, from 1 to N, or you can alternatively use the dendrogram for a more visual approach.
You can do so by considering each vertical line in the dendrogram and finding the longest line that is not bisected by a horizontal line. Once you find this line, you can draw a dotted line across, and the number of vertical lines crossed represents the number of clusters generated.
The longest line not bisected by a horizontal line has been colored orange. K-means and hierarchical clustering are both very popular algorithms, but have different use cases.
Hierarchical clustering, on the other hand, does not work well with large datasets due to the number of computations necessary at each step, but tends to generate better results for smaller datasets, and allows interpretation of hierarchy, which is useful if your dataset is hierarchical in nature. June 18, Data Basics Katie Gross. For more information on understanding key data science concepts as well as pros and cons of the most common machine learning algorithms, check out this detailed guidebook with the fundamentals.
Get the Guidebook.
0コメント