memory and run time for large dataset

[racng]

memory and run time for large dataset

[brianhie]

Hi @racng, sometimes Scanorama won't try to transform the values of two datasets, so it will be less than or equal to N choose 2 (where N is the number of datasets). Scalability will be with N choose 2, i.e., O(N^2) in the number of datasets.

In addition to batch size, you can also use the sketch parameter to the integrate() function and set the sketch_max parameter to get down to a total of 30k cells. Scanorama will still integrate all cells, but only make alignments using the sketch. That will hopefully let you integrate without having to pay for more compute.

Will close the issue since it's not a bug with Scanorama, but am still happy to answer any additional questions.