large sparse matrix as input

[teichlab]

large sparse matrix as input

[msmicker]

Hello, I am also interested in this. I am dealing with very large datasets. While it is possible to create a dense matrix I am curious about the limitations for cellphonedb to analyze even with subsampling. Is there experience with very large datasets? e.g. more than 100k cell? While I am able to export all of my datasets in dense matrix it is time consuming and also failing with any datasets larger than ~150k cells.

[Dzhan4]

Also curious about this. I was going to try to save sparse matrix as.data.frame(as.matrix(sparsematrix)) into a .txt file. I imagine this will take a long time. Anyone else have any ideas?

[rrydbirk]

I'm also listening in on this. It doesn't seem reasonable to write to .txt from a sparse matrix, in my case with ~3.4 billion fields...

[bjstewart1]

ideally both sparse matrix and anndata input rather than needing to store as .txt

[J-Keding]

I had the same issue with my sparse matrix that I extracted from a seurat object. I can reccomend looking at the write_dgCMatrix_csv function from the scrattch.io package, which writes the dgCMatrix to a CSV in chunks, thus saving time and memory.

[onahman]

+1 from me. It would be great to if you allow direct transfer of dataframe / matrices !
I tried to change the code myself but the FLASK API is a bit over my head

-- update--
I was able to run without the command line interface and it spared me the time of writing/reading of files. Still had a memory problem because my initial data was from R (Seurat) and I couldn't find a way to transfer it to python without converting it to a full matrix