sintel-dev/sigpro

Signal Processing Tools

sigpro's Issues

Update primitive interfaces

Currently SigPro primitives are stored as json files and used in pipelines as specifically formatted dictionaries. We would like to update this interface to be more object oriented, allowing for more intuitive construction of SigPro pipelines.

Accepting single value data frame format

Currently, dataframes are expected to bin values in each row. However, in a typical scenario, the data is expected to be recorded at each timestamp with one value. process_signal needs to handle the other type of data format.

The means that our dataframe goes from this expected format

  turbine_id        signal_id           timestamp                                             values
0       T001  Sensor1_signal1 2020-01-01 00:00:00  [0.43616983763682876, -0.17662312586241055, 0....
1       T001  Sensor1_signal1 2020-01-01 01:00:00  [0.8023828754411122, -0.14122063493312714, -0....
2       T001  Sensor1_signal1 2020-01-01 02:00:00  [-1.3143142430046044, -1.1055740033788437, -0....
3       T001  Sensor1_signal1 2020-01-01 03:00:00  [-0.45981995520032104, -0.3255426061995603, -0...
4       T001  Sensor1_signal1 2020-01-01 04:00:00  [-0.6380405111460377, -0.11924167777027689, 0....
..       ...              ...                 ...                                                ...

to something like

    turbine_id        signal_id           timestamp    values
0         T001  Sensor1_signal1 2020-01-01 00:00:00   0.43617
1         T001  Sensor1_signal1 2020-01-01 00:00:01 -0.176623
2         T001  Sensor1_signal1 2020-01-01 00:00:02  0.056531
3         T001  Sensor1_signal1 2020-01-01 00:00:04  0.433728
4         T001  Sensor1_signal1 2020-01-01 00:00:06 -0.019152
..         ...              ...                 ...       ...

Update HISTORY.md to show the latest updates

Update HISTORY.md to show the latest updates in SigPro.

Support for more complex pipeline architectures

Currently, SigPro supports only linear pipelines:

We would like to support _tree_shaped pipelines, which apply several primitives at each step to the values generated at the previous step:

Ideally, we can also generalize the previous example to support layered pipelines with a custom set of primitives applied to transform each layer to the following layer.

Incorrect classification for stft.json and stft_real.json

The stft and stft_real methods are listed in the JSON annotations under sigpro.transformations.frequency.stft.stft andsigpro.transformations.frequency.stft.stft_real, respectively. However, these primitives are actually implemented in the correct locations sigpro.transformations.frequency_time.stft.stft and sigpro.transformations.frequency_time.stft.stft_real. The JSON annotation should be amended to reflect this.

Support Python 3.9 - 3.11

Sigpro does not currently support python 3.9 through 3.11.

Syntax for pipeline creation

• SigPro version: 0.1.2
• Python version: 3.8
• Operating System:

Description

As it stands, it is not clear what a sigpro.SigPro object actually is from the name alone. Currently, the approach to constructing and running a pipeline is to instantiate a sigpro.SigPro object:

mypipeline = sigpro.SigPro(transformations, aggregations, keep_columns = True)

Following this, we can run the pipeline on sample data using the process_signal instance method, which under the hood calls the _apply_pipeline method:

processed_data, feature_columns = pipeline.process_signal(dataframe, '1h', time_index = 'timestamp')

However, the sigpro.SigPro constructor itself calls the _build_pipeline method to actually build the pipeline. A more intuitive interface would look like:

mypipeline = sigpro.create_pipeline(transformations, aggregations, keep_columns = True)

This new create_pipeline method should still return a SigPro pipeline (potentially not a sigpro.SigPro object), and should preserve the process_signal interface.

Loading demo dataset at random index

Since randint(a, b) generates a random integer a <= x <= b inclusive.

We should sample at most len(df) - 1

SigPro/sigpro/demo.py

Line 76 in 1a33006

index = random.randint(0, len(df))

Two-Column Data Format

SigPro should enable support for primitives that take two-column datasets (xvalues and yvalues). Currently, SigPro pipelines support only a single column of amplitude values.

Demo Notebooks for Pipeline usage

We would like to add a demonstration of the new Primitive and Pipeline classes as a notebook within the tutorials folder.