Is dates.json mandatory?

[vsainteuf]

Is dates.json mandatory?

[VSainteuf]

Hi @Depanshu-Sani,
dates.json is used in my code to compute the positional encodings based on the observation dates instead of the position in the sequence.
It is accessed in line 70 of dataset.py and used line 188 of train.py.

So a quick workaround for you would be to comment l.70-71 of dataset.py and instead initialise the variable d as an empty dict. You can then pass the --position order argument to train.py and this should work. However doing so, the positional encodings will be based on the sequence ordering instead of the observation dates.

You can also have a look at this issue where I point to more recent code that can handle time series with different date sequences.

[Depanshu-Sani]

Positional encoding based on the sequence order is exactly what I needed!
Thank you :)

[Depanshu-Sani]

I tried modifying the code as you mentioned but it seems that this not working. I commented line 70-73 (Ln 72 & 73 use the dates.json loaded in ln 70-71). I used the following command:

python train.py --dataset_folder ../Dataset/spatio-temporal/L8/SEASON_WISE/ --num_classes 2 --mlp1 [7,32,64] --mlp2 [128,128] --mlp4 [128,64,32,2] --geomfeat 0 --input_dim 7 --res_dir spatio-temporal/L8/SEASON_WISE --position order

Error:

Traceback (most recent call last):
  File "train.py", line 309, in <module>
    main(config)
  File "train.py", line 199, in main
    model = PseTae(**model_config)
  File "/home/depanshus/pytorch-psetae/models/stclassifier.py", line 26, in __init__
    T=T, len_max_seq=len_max_seq, positions=positions)
  File "/home/depanshus/pytorch-psetae/models/tae.py", line 47, in __init__
    positions = len_max_seq + 1
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

[VSainteuf]

Yes I forgot one thing: if you are not going to provide the list of dates, you need to specify the maximum length of sequence in your dataset (this does not have to be exact, an overestimation will do). This is done with the --lms argument of the train.py script.

[Depanshu-Sani]

It worked! Thanks for the quick response :)

[manapshymyr-OB]

@VSainteuf Hi, in this situation (where dates are not equal for the files) how should the normalization be calculated? Because the normalization has txc shape which t is time. I am a little bit confused. I am asking this because my dataset has not have all data for the dates extracted from my data

[manapshymyr-OB]

@Depanshu-Sani how did you calculate normalization if you do not have specific list of dates?