Bug Report: cluster: Possible errors occurred with LSH.

[skyhover]

Bug Report: cluster: Possible errors occurred with LSH.

[sail-amir]

@alex-from-intuita
As far as I remember, this is related to the limitations of one of the bash commands used in the tool. As for me, I ended up using a higher similarity threshold to deal with that. Also, plz visit our lab website at https://sail.cs.queensu.ca/ to get more information about the state of our research. I am particularly studying the quality of smart contracts.
Best,
Amir.

[skyhover]

Error: the structure supports at most 2097151 points (3238525 were specified).
The error is an inherent limitation in the LSH library used in Deckard; it can't handle more than 2million vectors at a time.

Using 0.79 similarity is not recommended as it often leads to many false positives.
Use a higher similarity, say 0.90.

Or as another alternative, split your input dataset into smaller ones before feeding it into Deckard. Then, after getting clone results for every smaller dataset, de-duplicate the vectors that have been identified as clones into one and then merge all the vectors left into one dataset to run Deckard again (need to write your own scripts for these, and there may be false negatives due to the split/merge process).

[alexbit-codemod]

@sail-amir i'd love to know if you could get this to work and am curious to learn more about your research! let's stay in touch! (linkedin)