Comments (7)
Hi there, I think I'm having the same issue as well.
If this is useful to anyone, I found that one possible workaround is to change 2 lines in densmap_.py (1737 and 1852 in the version I'm currently using) that seem to specifically test whether the data has 4,096 or more rows:
if X.shape[0] < 4096:
(at 1737)
if y.shape[0] < 4096:
(at 1852)
I changed them to some arbitrarily high number greater than the number of rows in my matrix/data frame and it seemed to run in 1 minute or less for my 5000 x 10 data matrix.
I did have to delete/refresh the precompiled versions of densmap_.py in the 'pycache' folder for it to take effect, though.
from densvis.
That's odd ... will look into it shortly, thanks!
from densvis.
Hi @rrydbirk so sorry that it took me so long to get to this. I've been looking into the issue to see if the entire R wrapper was compromised because of an update we made, but that does not seem to be the case. Do you mean that it only works for you when you input a square matrix? Would you be able to send me (at least a subsample) of the input data you are trying to use?
from densvis.
@ashwinn226 I just run the example codes in python3, It seems that the number of rows in the input array cannot exceed 4000, otherwise densmap will report the error,
import densmap
from sklearn.datasets import fetch_openml
from sklearn.utils import resample
digits = fetch_openml(name='mnist_784')
subsample, subsample_labels = resample(digits.data, digits.target, n_samples=7000, stratify=digits.target, random_state=1)
emb, ro, re = densmap.densMAP().fit_transform(subsample[0:3000,:])
emb, ro, re = densmap.densMAP().fit_transform(subsample[0:4000,:])
until this line,the program reports no errors
emb, ro, re = densmap.densMAP().fit_transform(subsample[0:4100,:])
Traceback (most recent call last):
File "", line 1, in
File "/Users/lvdekang/densvis-master/densmap/densmap/densmap_.py", line 2027, in fit_transform
self.fit(X, y)
File "/Users/lvdekang/densvis-master/densmap/densmap/densmap_.py", line 1794, in fit
self._search_graph.transpose()
File "/Users/lvdekang/miniconda3/envs/site/lib/python3.7/site-packages/scipy/sparse/lil.py", line 437, in transpose
return self.tocsr(copy=copy).transpose(axes=axes, copy=False).tolil(copy=False)
File "/Users/lvdekang/miniconda3/envs/site/lib/python3.7/site-packages/scipy/sparse/lil.py", line 462, in tocsr
_csparsetools.lil_get_lengths(self.rows, indptr[1:])
File "_csparsetools.pyx", line 109, in scipy.sparse._csparsetools.lil_get_lengths
ValueError: Buffer has wrong number of dimensions (expected 1, got 2)
from densvis.
Wow great find ... I have no memory of putting those lines in there, will look into what's going on.
from densvis.
Also facing this issue, looks like behavior is caused by differences between implementations of nearest neighbors search between small and large datasets.
from densvis.
Thanks all for patience. There was some issue with scipy's LIL sparse matrix type, and I just switched to CSR matrices, and this seems to have fixed the issue.
from densvis.
Related Issues (12)
- where is requirements.txt HOT 2
- Integration with UMAP repository HOT 2
- Distribution as R package HOT 6
- How can I get the cluster information of cells after densvis processing? HOT 1
- ZeroDivisionError: division by zero HOT 2
- prediction on out of sample HOT 1
- Can densMAP (in development version of Seurat) yield reproducible clustering? HOT 3
- How to find the quantitative evaluation of density preservation? HOT 2
- Can I know the simulation settings? HOT 2
- Support for Numba 0..55.2
- Install issue on apple silicon
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from densvis.