Face clustering runs Out of memory

[blackmore190]

Face clustering runs Out of memory

[jw1720]

@blackmore190 I had the same problem. I looked at the code that the error message pointed to, it seems the problem is that the program needs to allocate memories for all possible pairs of images (unless it is too different based on the Euclidean distance) in order to run the Chinese Whisper algorithm. This doesn't work for large number of images.

Without a change on how the algorithm works, the only thing we can do is to lower the clustering threshold (in the admin setting panel). I have 118k images, 54k faces and need to low the threshold to 0.32 in order to complete the clustering in a machine with 11G RAM. The result is still pretty good.

I do think the current implementation of algorithm is not scalable to very large number of images. So I hope a new algorithm can be used to address the issue.