Inquiry on Utilizing UMAP for Text Similarity and Clustering

[dsdanielpark]

Inquiry on Utilizing UMAP for Text Similarity and Clustering

[lmcinnes]

Thank you for the kind words.

Mostly what UMAP will buy you over using DBSCAN directly on the embedding vectors is a lot more of your data clustered while still having reasonably fine-grained clusters. Can I guarantee better results? I think there are no guarantees, especially in unsupervised learning. Would I expect better results if you use UMAP first and then DBSCAN or HDBSCAN? Yes, I definitely would.

Choosing parameters is always going to come down to the data you have, the kinds of results you want to get, and what you are going to use the clustering for from there. Some rules of thumb: n_components=5 is a good starting point for clustering. It is enough dimensions that UMAP has a much easier time resolving tangles etc. in the optimization, but still pretty low. I would not choose n_components larger than n_neighbors (or really larger than 20 even if you have a very large n_neighbors). The choice of n_neighbors is going to strongly influence the granularity of the clustering. The smaller the value the more fine grained the resolution of clusters you'll tend to get out (assuming DBSCAN or HDBSCAN for clustering the UMAP output). As for metric, the usual choice for sentence embeddings is "cosine"; if you want to try something a little different then import pynndescent and use pynndescent.distances.alternative_cosine which is a small tweak on cosine distance that may work better for your use case with UMAP.

[dsdanielpark]

Thank you for your kind response. I'll start as you suggested!

Can UMAP be updated in batches? Is it possible to create a UMAP model for large images and further train it? It seems impossible due to UMAP's mechanics, but I wonder if implementing this feature would be difficult.

[lmcinnes]

I think for that use case you might want to look into ParametricUMAP. UMAP does have an update method, but it is definitely not the same as training on the full dataset.