Kmeans failed to converge in 1m iterations

4 views (last 30 days)
Hi all
I'm using Kmeans++ to classify a 1 000 000 x 6 dataset. After shrinking those data into [0, 1] (or zero-scored), and setting the MaxIter from 1000 to even 1m,
with
[idx, C, sumd, D] = kmeans(metricsStd,k, 'Display','final', 'Replicates', 100, 'MaxIter',10000000);
I still keep getting warning saying:
Replicate 96, 79 iterations, total sum of distances = 41713.9.
Warning: Failed to converge in 10000000 iterations during replicate 96.
Replicate 97, 66 iterations, total sum of distances = 41652.
Warning: Failed to converge in 10000000 iterations during replicate 97.
..... and on and on.
I tried various numbers of Replicates, MaxIter, ks (from 2 to 10 for example), and tried standardization and normalization. It keeps showing this.
I'm wondering if there's anything I've missed? Or does it mean that my data themselves are non-convergent?
Add1: I picked 3 dimensions of my data. It seems that there're no clusters at all or just tiny clusters that cannot be seen. It's from a biomedical experiment and from our 'experiences', there 'could' be some clusters that people can't possibly manually figure out. It's like we have firm evidences that they can be separated.
  2 Comments
Image Analyst
Image Analyst on 27 Jan 2022
Edited: Image Analyst on 27 Jan 2022
Please attach a screenshot of 3 of the dimensions plotted with scatter3() so we can see if there are any clusters. Are you sure there are clusters and it's not just all random? Attach your data if you need more help, after you re-read this:
Huanchen Cai
Huanchen Cai on 27 Jan 2022
Thanks for your advice. I added a pic in the question. It's like from expert experiences there should be some of clusters so we would like to see if kmeans can pick them up.

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!