# 1. What Is K-Means Clustering
K-means is an unsupervised clustering algorithm that groups data points into distinct clusters based on their similarity, with no labeled data required.
# 2. The K-Means Algorithm
K-means works through an iterative four-stage process:
1. **Initialization.** Choose the number of clusters to create — this is the "K" in K-means — and randomly place K centroids (cluster centers) within the data space.
2. **Assignment.** For every data point in the dataset, calculate its distance to each of the K centroids, and assign the point to whichever cluster's centroid is closest.
3. **Centroid update.** For each cluster, recalculate its centroid as the mean (average) of all the data points currently assigned to it.
4. **Repeat.** Steps 2 and 3 repeat until the centroids stop moving significantly between iterations, or a maximum number of iterations is reached.
```
Function KMeans(data, K):
Randomly initialize K centroids, c1, c2, ..., cK
Repeat:
for each data point x in data:
Find the nearest centroid, ci, to x
Assign x to cluster i
for each cluster i = 1 to K:
Update centroid ci to be the mean of points in cluster i
Until convergence (centroids don't change much, or a max number of iterations is reached)
```
# 3. Important Considerations
**Determining K.** Choosing the right number of clusters is one of the least trivial parts of using K-means — methods like the elbow method or silhouette scores (see the Data Science and ML Knowledge note) can help identify a sensible value.
**Initialization sensitivity.** Because centroids start at random positions, K-means can converge to different clustering solutions on different runs. Running the algorithm multiple times with different random starting points, and keeping the best result, helps guard against landing on a poor local optimum.
**Distance measures.** Euclidean distance is the most common choice, but other distance metrics — such as Manhattan distance — can be more appropriate depending on the structure of the data.
# 4. Why Scaling the Data Matters
K-means typically uses Euclidean distance to assign each point to its nearest centroid, and centroid positions are themselves computed as an average — which makes the whole algorithm sensitive to outliers. For example, given a feature set of 1, 2, 3, 3, 4, and 10, the mean without the outlier (10) is 2.6, but including it pulls the mean up to 4.8. Outliers like this distort the resulting cluster quality, which is why scaling the data is essential — though it's worth noting that not all scaling methods handle this equally well, since some (like standard scaling) still rely on averaging internally. **Robust scaling**, which uses quantiles instead of the mean, is a better choice when outliers are a concern.
Scaling also matters for a second reason: when two features differ greatly in range, the centroid positioning gets pulled toward whichever feature has the highest variance. For example, suppose feature $X_1$ ranges from 5 to 6 between its 10th and 90th percentiles, while feature $X_2$ ranges from 5000 to 16000 over the same percentiles — with $X_1