Comments (15)
Possibly related to joblib/joblib#849
from giotto-tda.
Hi,
Thanks for the feedback. Could you provide your joblib version?
from giotto-tda.
joblib 0.14.0
Seems to be the most recent version.
from giotto-tda.
@arksch was this ever resolved? Is it still happening with the latest version of giotto-tda
?
from giotto-tda.
I'm getting this issue as well.
from giotto-tda.
@jlevy44 thanks for the report! Could you please provide us with your setup and with a minimum working example so we can try to reproduce this issue?
To get the setup, please run the following snippet and paste the output:
import platform; print(platform.platform())
import sys; print("Python", sys.version)
import numpy; print("NumPy", numpy.__version__)
import scipy; print("SciPy", scipy.__version__)
import joblib; print("Joblib", joblib.__version__)
import sklearn; print("Scikit-learn", sklearn.__version__)
import gtda; print("Giotto-tda", gtda.__version__)
from giotto-tda.
Sounds good.
Linux-3.10.0-1062.18.1.el7.x86_64-x86_64-with-debian-buster-sid
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0]
NumPy 1.19.1
SciPy 1.5.2
Joblib 0.16.0
Scikit-learn 0.23.2
Giotto-tda 0.2.2
Instead of posting a working example, I can send you my solution, which I was able to get to run! I ended up hacking your script and replacing the joblib backend with dask:
from gtda.homology._utils import _postprocess_diagrams
import dask, dask.diagnostics
from gtda.homology import VietorisRipsPersistence
homology_dimensions = (0, 1)
def fit_persistence(x):
VR = VietorisRipsPersistence(
metric='euclidean', max_edge_length=3000, homology_dimensions=homology_dimensions, n_jobs=1).fit([x])
return VR._ripser_diagram(x)
with dask.diagnostics.ProgressBar():
diagrams=dask.compute(*[dask.delayed(fit_persistence)(x) for x in point_clouds],scheduler="processes")
diagrams=_postprocess_diagrams(diagrams,(0,1),None,1)
This worked as intended. :)
from giotto-tda.
@jlevy44 thanks for reporting on your setup and it's interesting that you found a solution with dask!
However, it is still important to have a minimum working example (if you are able to provide it), to be able to better pinpoint the exact origin of the problem. Since we use joblib
for now, we cannot fix the issue otherwise!
from giotto-tda.
Additionally, I'm wondering if you were passing a collection with a single sample even before finding the solution with dask.
from giotto-tda.
No, I wasn't, if I'm understanding you correctly. Sometimes it (joblib backend) would work (especially if I reduced the number of point clouds to ~30), other times it would fail (especially close to 100 clouds and higher max_edge_length).
I am unable to provide our working example right now, apologies!
from giotto-tda.
@jlevy44 thank you, I understand that you cannot provide this example. Could you perhaps give me information on the shape of the point clouds you use?
from giotto-tda.
@jlevy44 aside from my above question about shapes and number of point clouds used (so I can try my best to reproduce), I'm wondering if you can try the following since you don't seem scared by the idea of changing the source code: in the call to Parallel
in VietorisRipsPersistence.transform
, could you try to insert various allowed values for the keyword parameter backend
, according to https://joblib.readthedocs.io/en/latest/generated/joblib.Parallel.html, after n_jobs
? It would help greatly to know if this improves the situation or not.
from giotto-tda.
@arksch was this ever resolved? Is it still happening with the latest version of
giotto-tda
?
I cannot reproduce it with the newest version. Solved from my side. Thanks!
Linux-4.15.0-117-generic-x86_64-with-Ubuntu-18.04-bionic
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0]
NumPy 1.19.2
SciPy 1.5.2
Joblib 0.16.0
Scikit-learn 0.23.2
Giotto-tda 0.2.2
from giotto-tda.
@arksch thank you so much for coming back to this thread, and for the report!
It would be good to have @jlevy44's answer so we can attempt to reproduce his issue and improve the experience for other users.
from giotto-tda.
Closing as this is likely no longer an issue with the giotto-ph
backend.
from giotto-tda.
Related Issues (20)
- Correlation dimension
- gtda/mapper/tests/test_visualization.py tests failed HOT 5
- Giotto Mapper crashing on large data
- Unexpected Cech persistence result returns
- Prebuilt wheel for Python 3.11 HOT 3
- Boundary of homological features
- [BUG]
- [BUG] Installing giotto-tda error HOT 5
- [BUG] HOT 2
- [BUG]
- simplex index 9223716361969802160 in filtration is larger than maximum index 36028797018963967
- [FEATURE REQUEST] GPU-support with CuPy
- Scikit-Learn 1.3 support HOT 1
- [BUG]
- [BUG] Cannot run Scaler() HOT 3
- [BUG] can't download Giotto HOT 1
- Installation problem
- [BUG]: Cannot install Giotto-tda: Failed to build giotto-tda ERROR: Could not build wheels for giotto-tda, which is required to install pyproject.toml-based projects HOT 9
- No visualization after plot_diagram HOT 7
- [BUG] ERROR: Could not build wheels for giotto-tda-nightly, which is required to install pyproject.toml-based projects
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from giotto-tda.