martinheroux/spike2py

spike2py provides a simple interface to analyse and visualise data

Hello and thank you for visiting my Github page!

My name is Martin Héroux and I am currently based in Sydney, Australia.

I have been coding for over 15 years and love to learn about computer science and the various concepts and practices that underpin clean code and maintainable software.

Scientist

I am a Research Fellow at Neuroscience Research Australia, where I work on a variety of exciting projects ranging from sensory illusions and muscle mechanics to motoneuron physiology, electrical stimulation to restore walking after spinal cord injury and various topics related to open and reproducible science.

You can find my scientific publications on ResearchGate and Google Scholar.

I maintain the scientificallysound blog with with my colleague and friend Dr. Joanna Diong. We discuss news and trends related to open and reproducible science and provide tutorials on statistics and coding.

Sample repositories related to my research:

Python Developer

I am passionate about Python and its community. I have been using Python for all of my personal and professional projects for several years now.

I have blogged about Python on scientificallysound and was fortunate enough to be a guest on the TalkPython podcast.

I have published two packages on PyPi related to tools I have created as part of my research:

Human

I enjoy exchanging ideas with like-minded individuals. Feel free to reach out and start a conversation.

spike2py's Issues

Importing issues

Hi bob,

I am having trouble finding a solution to my problem.

I have modified my __init__.py in the main package folder.

The reason I want to do this is to limit the user interface when using the package. Before this change, importing spike2py meant that spike2py was full of functions that I did not want to user to necessarily have access to (e.g. spike2py.mixin).

The changed __init__.py

from channels import Channel_Details, Event, Keyboard, Waveform, Wavemark
from trial import Trial_Info, Trial

With the new version of the __init__.py file, the user is not distracted by lots of functions and classes that are not meant for the end user. See this screenshot of what it looks like:

But doing this means that I can't run my tests. I guess this is because I previously had all the modules imported into to __init__.py spike2py file. I tried a few hacks that I found on stack overflow but none of them worked.

Have you ever come across this? Do you know how to fix it so that I can run the tests but not have the user see/access all the methods and classes?

Code style and type hinting

I have heard a lot about flake8, and more recently black.

What are your views on black? It seems like a sensible thing to further increase the look and readability of code, and there seems to be ways to include running black (and other tools) as part of the commit process. A bit over my head, but I was looking at this post on the topic.

---

As mentioned elsewhere, I like the idea of type hinting, but am not sure how to implement it consistently (and to what extent).

As we work more on my code, let me know if you can see where/how I could include type hints. Should they be everywhere? Is it worth the effort?

Documentation - Sphinx, jenkins, etc

Here is the post I mentioned during our call.

I used github pages to host the documentation of the first version of my spike2py packages. But I had to remember to go and regenerate the documentation manually every time I made additions or changes.

I did not have to write a jenkins file or anything. I simply followed a 3-part youtube video series of something walking me through the steps. But I would love to find a way to automate documentation generation/update (for example, after any merge of a pull request).

---

Do you see it as useful to include examples in user-facing classes, methods, functions?

For example, in the previous version of the spike2py package, I adopted a style I copied from ?? numpy...

"""Calibrate signal.
        Done when signal first created if `calibInfo` parameter provided.
        Calibrated signal will be available as `.proc`. However, it
        can also be performed after the signal has been created if the
        `slope` (and `offset`) parameters are provided. In this case, the
        calibration is applied `.proc` and stored in-place. `calibInfo`
        will be added to sig.
        See `CalibInfo` for details on `slope` and `offset`.
        Examples
        -------
        >>> import spike2py
        Calibrate signal after it has been created; slope only. Show `calibInfo`.
        >>> trial.sig['pressure'].calibrate(slope=-2.17)
        >>> trial.sig['pressure'].calibInfo
        CalibInfo(slope=-2.17, offset=None)
        Calibrate signal after it has been created; slope and intercept.
        Show `calibInfo`.
        >>> trial.sig['pressure'].calibrate(slope=-2.17, offset=-1.8)
        >>> trial.sig['pressure'].calibInfo
        CalibInfo(slope=-2.17, offset=-1.8)
        """

This would look nicely formatted when read in by Sphinx. But it does add a big chuck of text between a class/method declaration and the actual code. So it will help the user of the code, but might be distracting to someone working on the code itself.

Any thoughts?

app mindset / structure

Pasting mindmap here to have it handy as we work through this project together:

So you start with the data processing side right? Please commit some files to tests/payloads

Method chaining

Hi Bob,

You may have already found out all about this, but method chaining seems very simple.

Method chaining

The one thing I am wondering about is that they are methods that are being called; that is, each of my signal processing methods have to be part of the class that I am using chaining with (i.e. Waveform).

How would this work if I had all my signal processing methods in another class, possibly a mixin?

I guess that now leads me to wonder what the str class and others look like inside....do they have lots and lots of methods in their class? Or is there a nice way to make the methods of one class (e.g. a mixin) appear to be the methods of another class (e.g. waveform).

I will look into this, but thought I would ask if you had any thoughts....

[FEATURE]: Spike2py alias and comments

Hi there, thanks for a neat package -- it makes importing data from Spike2 much more streamlined. I had some comments that seem more suited as a Discussion instead of as an Issue, so Feature seemed the best fit.

Spike2py alias
The Tutorial recommends importing spike2py using the alias sp, however scipy is often imported as sp. I can imagine some users might want to use scipy functions after having imported data using spike2py, which would create conflicts. What do you think about importing spike2py using a different alias, e.g. s2p, as a recommendation?

Demo data file location
In the Tutorial, when practicing loading the pickle file, I could not work out where demo tutorial.mat file is located to supply the file path of the pickle file to run the code. If the intention is for users to test storing and loading the files at this point, could the docs advise where the tutorial.mat file is stored?

Hyperlinks to functions
In the How To Guides, at the list of signal processing steps, the docs said clicking on the names takes users to the full documentation but the names are not hyperlinked. I'm not sure if they should be hyperlinked, or whether the docs refers to clicking on function names in an IDE. Could this be clarified?

Thanks for your consideration.

Including tutorial data and possible issues with pypi build

I thought the best way to provide an error-free tutorial experience would be to ship a sample dataset with spike2py. That way everyone will be able to test their installation as well as work through the tutorial without having to have their own data.

The potential problem with this approach is that my tests folder is beside the spike2py folder, not inside of it. I guess there are two approaches to pytest and where the tests folder is located. In reading more about pytest, etc, it seems that the current folder structure might mean that the tests folder will not be part of the package uploaded to pypi.

This is a problem because the sample dataset is actually one of the datasets in the payload folder, which may not be included in the pypi build.

Should I consider moving the tests folder inside of the main spike2py folder (where the actual code lives)? Or should I include the sample dataset alongside (or in a demo folder) located in the spike2py folder with the modules?

Documenation update

Apologies! I did lots of work today and did not realise I was on the master branch.

As it is getting late I could not figure out quickly how to move all of this work to a new branch so that could push and make a pull request.

Therefore I simply pushed it to the master branch. Thankfully, most of the changes/additions were to documentation. I have done lots of work on it and have completed the tutorial.

Have a look: firefox (or chrome) spike2py/docs/builds/html/index.html

There is more to do, but I feel I have a good grasp of what needs to be done and I would rather move on to the next project and not spend even more time documenting the project.

Having said this, I don't want to forget about this project because I do want to figure out how to use pre-commit hooks (black, pytest) and also how to get my package (once I finish the documentation), up on pypi.

add a requirements.txt file

I guess I need this, please add the pinned libraries in a requirements.txt file. Thanks!

[bobbelderbos@imac spike2py (test_read)]$ agg import
tests/test_read.py
1:import os
2:import pytest
3:import numpy as np
4:import scipy.io as sio
6:import read

spike2py/read.py
1:import sys
2:import textwrap
3:import scipy.io as sio

Add test for init portion of trial.py

sonpy dependency

As discussed today, as the library is old (py 2) and not actively maintained, best to work on a fork and ultimately get that on pypi, so you can pip install it into this project (not clutter this project with this external code).

I am glad you are focused on the big picture though, realizing that this was too much of an effort right now.

I opened this issue merely for tracking.

Bob

Minor fixes needed to read.py

1. Check if wavemark channel actually has data/times present. Current, the code will crash if a wavemark channel is present but contains no data.

2. L25 current -> file_extension = file.split('.')[1]
   replace with -> file_extension = os.path.splitext(file)[-1]
   (need to import os; also need to consider that file_extension will now include the dot)

Write tests for read.py module

Importing issues

I am a bit at a lose with the importing stuff.

package/
    __init__.py
    module1.py
    module2.py

I was under the impression that if package was part of the Python path, and __init__.py was empty, I should be able to do the following:

import package
package.module1.fx_in_module1()

from  package import module1
module1.fx_in_module1()

from  package.module1 import fx_in_module1
fx_in_module1()

This does not work with spike2py. I can run import spike2py but no modules or functions appear.

Weirdly, if I add from . import trial to __init__.py, I can run import spike2py and all the modules appear, not just trial

**Similar if I use from .trial import Trial, TrialInfo

This asses these Classes, in addition to the modules. And each of the modules will include all the variables and imports (e.g. np)

---

PEP328

Similar question on stackoverflow about namespaces (like np showing up). Also talks about all

Keneith Reitz on modules/package

pydocs good example with package

pytest-mpl

There is a matplotlib testing plugin for pytest.

Definitely will be worth looking into and adding it to spike2py.

TO DO list for spike2py - reminder of things that need to be finished

• Create tests for aspects of plot.py that can be tested
• Create tests to confirm plots are being generated and saved
• Add how-to's to the documentation:
  • For a more detailed TrialInfo prior to importing
  • For each of the methods in the sig_proc mixin class
• Topic guides
  • Why the choice to use data exported to .mat?
  • Scope: what spike2py is and what it isn't
• Update current tests
  • Have error tests use 'match' to confirm that the proper error term is being returned
  • Use 5.13434322311 == approx(5.1) for testing numbers with decimal points (i.e. approx rather than hack I am currently using
• Look into pytest-mpl for a possible way to add tests for generated figures in matplotlib

[BUG] trial plotting not working when all signals are events and keyboard channels

Read in data from a single event channel and the keyboard channel, then try to plot the summary plot that includes all channels.

Currently this throws an error.

current_trial.plot()
Traceback (most recent call last):
File "", line 1, in
File "/home/martin/Dropbox/Martin/Documents/research/projects/activeProjects/spinal_stimulation/SSASSY/pythonProject/venv/lib/python3.8/site-packages/spike2py/trial.py", line 142, in plot
plot.plot_trial(self, save=save)
File "/home/martin/Dropbox/Martin/Documents/research/projects/activeProjects/spinal_stimulation/SSASSY/pythonProject/venv/lib/python3.8/site-packages/spike2py/plot.py", line 187, in plot_trial
f"The trial {spike2py_trial.name} has only one plottable channel."
AttributeError: 'Trial' object has no attribute 'name'

It is possible this should refer to spike2py_trial.info.name...

Is it customary to delete branches once they have been merged?

On my machine, I still have the init branch where I created two folders (spike2py and tests) each with an __init__.py file.

This branch had a pull-request logged on github and it was merged.

Should I leave the branch there, or can I delete it? I ask this because I am assuming that we will start to have lots of branches very soon.
If I am to delete the branch, where should this happen? On my machine, or github, or both? Or if I delete the branch in local will this be passed along to origin?