Giter Site home page Giter Site logo

Comments (2)

maartenbreddels avatar maartenbreddels commented on April 28, 2024

Actually it should, but only 'named' selections. For instance

ds.plot("x", "y", selection="z > 0")

would not cache anything, but

ds.select("z > 0", name="zpos")
ds.plot("x", "y", selection="zpos")

should cache it, so executing the plot a second time should go faster.
Although caching all selections with some memory limit would be an option, I want to leave it out to keep vaex as simple as possible. I hope this will suffice for the moment.

However, I was testing the performance of this, and found it was not being cached. I fixed this, however, don't expect massive speedups for simple selections. Since having a selection means that the data has to be copied (though this may change in the future), it means a performance penalty for selections.

I'll get back to you on this, in combinations with performance improvements for multiple plots, you may see some speedup, which should also fix #10.

from vaex.

maartenbreddels avatar maartenbreddels commented on April 28, 2024

This is working well now (using arctan2 since it's quite expensive to calculate):

ds = vaex.datasets.helmi_de_zeeuw.fetch()
ds.select("arctan2(y,x) > 0")
%%timeit -r5
ds.count(selection=True)

Gives:

10 loops, best of 5: 18.5 ms per loop

While using an unnamed selection (which will not get cached)

%%timeit -r5
ds.count(selection="arctan2(y,x) > 0")

Gives

1 loop, best of 5: 120 ms per loop

6.5x speedup.

from vaex.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.