glentner/dataphile

Data analytics library for Python and suite of open source, command line based data ops tools.

Dataphile

Dataphile is a high-level python package for both data analysis and data processing. It started as a central repository of common tasks and capabilities used by the author, but has now evolved into something others might find useful. See components below.

---

Figure: Demonstration of Dataphile's AutoGUI feature.

Installation

To install Dataphile for general purposes use Pip:

pip install dataphile

If you are using Anaconda, install using the above call to pip from inside your environment. There is not as of yet a separate conda package.

Documentation

Documentation will be available at dataphile.readthedocs.io. Currently, development of additional features is a priority, but this is a great place for contributing to the project.

Contributions

Contributions are welcome in the form of suggestions for additional features, pull requests with new features or bug fixes, etc. If you find bugs or have questions, open an Issue here. If and when the project grows, a code of conduct will be provided along side a more comprehensive set of guidelines for contributing; until then, just be nice.

Road Map

• additional command line tools
  Many additional command line tools are planned for future releases including tools that expose database queries and filters. Generally, just a massive extension of the UNIX philosophy whereby we can compose several functions together with pipes to make unique workflows.

• data acquisition
  One of the motivations for this package was to provide an easy-to-use, high-level interface to collecting scientific data from an externel device (e.g., over USB). This, along side a simple live data visualization feature would go a long way for high school and university student laboratory courses to both aquire and analyze their data using all open-source tools right inside of a Jupyter Notebook.

• documentation and package management
  A quickstart guide along with full documentation of all components needs to be built using Sphinx.

dataphile's Issues

Issue with importing ConsoleHandler from logalpha

Hi,

I'm having issues with running dataphile. I tried to run:

from dataphile.datasets import SyntheticDataset
from dataphile.statistics.distributions import polynomial1D

xdata, ydata = SyntheticDataset(polynomial1D, [100, -0.01, -1e-5], (0, 2400), linspace=True,
noise=0, samples=2400).generate()

I get the following error:

ImportError: cannot import name 'ConsoleHandler' from 'logalpha' (C:\anaconda\lib\site-packages\logalpha_init_.py)

I've tried to reinstall logalpha but the error persists.
I'm using python 3.7 in Anaconda

Thanks for your help!

GUI freezing

Hi!
Very nice tool you implemented!
On my mac, python 3.6 it is mostly working all right.
But once every 2-3 times, the gui (sliders) freezes and I did not manage do understand the reason (no warning etc).
It might be when the plot leaves the x/y limits of the window but sometimes even if it stays within the limits.
Did you already experienced this problem?
Best

Vincent

Updating data when fitting on a part of the plot

Hello Geoffrey!
I was thinking about a way to improve the fitting with your tool.
Indeed, what can be annoying is that when you have loaded some points you are obliged to fit them...
What do you think about fitting only the points that are displayed on the plot (taking into account the xlim/ylim of the figure).
This would mean that you can zoom on the part of the plot that you want to fit but also zoom out if you want to use again the whole dataset... in this way it is just a new feature that can not be harmfulll.

A naive way to implement it is just to add to modify change the line self.model.fit(*self.data) by the following in the autoGUI class.

What do you think about adding that?

    def __fit_button_on_clicked(self, event):
        """Action to take when the 'Fit' button is pressed."""
        # call the model.fit with data
        axis = self.figure.axes[0]
        mask = (self.data[0]>axis.get_xlim()[0]) & (self.data[0]<axis.get_xlim()[1]) & (self.data[1]>axis.get_ylim()[0]) & (self.data[1]<axis.get_ylim()[1])
        xdata = self.data[0][mask]
        ydata = self.data[1][mask]
        self.model.fit(xdata,ydata)
#        self.model.fit(*self.data)
        self.__update_graph()
        self.__create_sliders(self.active_model.label)

Best

Vincent