Fixing Tensor Shape/Dimension Mismatch Errors in TimeSeries Transformer for Stock Price Prediction

[sivavishnubramma]

Fixing Tensor Shape/Dimension Mismatch Errors in TimeSeries Transformer for Stock Price Prediction

[amyeroberts]

cc @kashif

[kashif]

thanks! having a look

[sivavishnubramma]

Appreciate it. Please let me know for any additional information that can help with the debugging.

[sivavishnubramma]

If I understand correctly, the issue is in this code snippet:

sample = {
   'past_values': self.X[idx],
   'past_time_features': torch.zeros((time_dim, feature_dim)), 
   'past_observed_mask': self.observed_mask[idx],
   'future_values': future_values,
   'future_time_features': future_values,
}

Appreciate your timely support!

[kashif]

@sivavishnubramma indeed the issue is in the sizes of the inputs... so what are the shapes of "past_values" and "future_values" tensors?

[kashif]

also note that the future_time_features are the known covariates int he prediction window (i.e. the known date-time covariates for example or any static covariates)

[sivavishnubramma]

I added a few time variant to the code. Here are the shapes of the parameter "

Sample 523 - past_values shape: torch.Size([10, 74]), past_time_features shape: torch.Size([10, 74]), past_observed_mask shape: torch.Size([10, 74]), future_values shape: torch.Size([10, 74])
Sample 186 - past_values shape: torch.Size([10, 74]), past_time_features shape: torch.Size([10, 74]), past_observed_mask shape: torch.Size([10, 74]), future_values shape: torch.Size([10, 74])
Sample 441 - past_values shape: torch.Size([10, 74]), past_time_features shape: torch.Size([10, 74]), past_observed_mask shape: torch.Size([10, 74]), future_values shape: torch.Size([10, 74])

fyi- Prior to adding the time variants, the shape was [10, 70]

[sivavishnubramma]

The error messages are somewhat challenging for someone new to this area to decode. I have a more specific question regarding dataset construction for the Time Series Transformed Model training:

1. I have a dataset from a CSV file with columns: {Date, Open, Close, Volume, Price_change, Bull_Bear_Signal}.
2. I've added the following time-related features: {'day_of_week', 'month_of_year'}, increasing the total feature count to 8, including the Date.
3. I then removed the Date column, leaving 7 features in the dataset.
4. The look-back period is set to 10, and the prediction length is 1.
5. The model outputs 2 predictions: {Price_change, Bull_Bear_Signal}.

Could you please guide me on how to correctly structure the following dataset to avoid previous errors and ensure successful model training?
(OR)
If there is a builder/adapter class that gets the user requirement similar to the above list and produces the dataset necessary to train the model, please let me know.

sample = {
    'past_values': <>,
    'past_time_features': <>,
    'past_observed_mask': <>,
    'future_values': <>,
    'future_time_features': <>
}

Let me know for any additional details.

[kashif]

@sivavishnubramma we do have a blog post explaining the inputs and what the batch samples should be here: https://huggingface.co/blog/time-series-transformers

did you manage to have a read of the blog post?