Similarity of histograms: interpretation of cosine and jaccard similarities with "pdist2"

8 views (last 30 days)
I would like to assess the similarity between two "bin counts" (that I previously derived through the "histcounts" function), by using the "pdist2" function:
% input
bin_counts_a = [689 430 311 135 66 67 99 23 37 19 8 4 3 4 1 3 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1];
bin_counts_b = [569 402 200 166 262 90 50 16 33 12 6 35 49 4 12 8 8 2 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 1];
% visualise the two "bin counts" vectors as bars:
bar(1:length(bin_counts_a),[bin_counts_a;bin_counts_b])
% calculation of similarities
cosine_similarity = 1 - pdist2(bin_counts_a,bin_counts_b,'cosine')
cosine_similarity = 0.9547
jaccard_similarity = 1 - pdist2(bin_counts_a,bin_counts_b,'jaccard')
jaccard_similarity = 0.0769
If the cosine similarity is close to 1, which means the two vectors are similar, shouldn't the jaccard similarity be closer to 1 as well?
  2 Comments
Dyuman Joshi
Dyuman Joshi on 26 Jun 2023
"If the cosine similarity is close to 1, which means the two vectors are similar, shouldn't the jaccard similarity be closer to 1 as well?"
No, because the similarities are defined differently. Cosine similarilty is not same as Jaccard similarity.
You can check out the definitions in the More About section of the pdist2 documentation page.

Sign in to comment.

Accepted Answer

Sai Teja G
Sai Teja G on 14 Aug 2023
Edited: Sai Teja G on 10 Oct 2023
Hi Sim,
I see that you are comparing two vectors by using cosine and jaccard distances between them.
They are not the same, as Jaccard Similarity considers a set of unique word lengths, while cosine similarity considers the entire sentence vector, disregarding data duplication.
Please refer the following documentation for more information on distance metrics like ‘jaccard’ and ‘cosine’ for the function ‘pdist2()’ –
Hope it helps!

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!