aqibsaeed/sensor-transformer

Transformer Network for Time-Series, Sensor and Wearable Data

Sensor Transformer (SeT)

Adaptation of Vision Transformer (ViT) for Time-Series and Sensor Data in Tensorflow.

Problems/Datasets

• Human Activity Recognition with Wearable Device
• Seizure Detection
• Predictive Maintenance

Tools

• Tensorflow 2.4
• einops

Install

pip install sensortransformer

Usage

import argparse
import tensorflow as tf
from sensortransformer import set_network

parser = argparse.ArgumentParser()
parser.add_argument("--signal-length", type=int)
parser.add_argument("--segment-size", type=int)
parser.add_argument("--num_channels", type=int)
parser.add_argument("--num_classes", type=int)
args = parser.parse_args()

"""
TF-Data objects, see data.load_data function.
Instances must be of shape x = (batch, signal_length, num_channels)
y = (batch, num_classes)
"""
ds_train, ds_test = ...

model = set_network.SensorTransformer(
        signal_length=args.signal_length,
        segment_size=args.segment_size,
        channels=args.num_channels,
        num_classes=args.num_classes,
        num_layers=4,
        d_model=64,
        num_heads=4,
        mlp_dim=64,
        dropout=0.1,
)
model.compile(
    loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
      optimizer=tf.keras.optimizers.Adam(),
      metrics=[tf.keras.metrics.CategoricalAccuracy()],
)
model.fit(ds_train, epochs=50, verbose=1)
model.evaluate(ds_test)

Thanks to Phil Wang for open-sourcing Pytorch implementation of ViT

sensor-transformer's Issues

Sample Dataset to test with

Hello
Thank you for yout great work developing this code.

I would like to know if there are any ready samples converted to npy file to test the code with.

Thank you

Low Accuracy and Signal/Segment Parameters

Hello Aqib

I ran the code the has previously linked to into the issue #1

After running the code, the generated training/testing X dataset is 4 dimensions (17111, 1, 90, 3) and (7292, 90, 3) respectively.

I am not aware if what I have done is correct, but I removed the "height" dimension from this line, so the sensorTranformer can to match the input requirments.

reshaped_segments = segments.reshape(len(segments), 1,90, 3)

to become

reshaped_segments = segments.reshape(len(segments) ,90, 3)

Regarding signal-length and segment-size, I am not aware of the values that shall be used. I know that the segment-size shall be less than or equal to 90 and as a divisor, the remainder shall be 0. What about the signal-length.

I expirmennt with random values for signal-length and segment-size, 90 and 10 respectively for 50 epochs uwsing GELU activation, however, the resultant accuracy is 0.0957, which is very bad. RELU was much better at around 0.45. What was a bit strange for me is that Swich was around 0.0974, which is as bad as GELU. This is leaving me scratching my head in confusion.

What am I doing wrong here?
Thank you in advance