cchallu/nbeatsx

nbeatsx's Issues

`run_test_nbeatsx` confusion

I am trying to use this project on my own task. I have gone through the process to tune a model and find an optimal configuration.

However, during the process of trying to forecast on future data, I am getting significantly worse MAE than during training/testing.

I am confused about run_test_nbeatsx and its behavior. My understanding is that the exogenous variables for the forecast horizon are not available at prediction time. However, when I change the values in X_df when forecasting, the output y forecast values change.

In run_test_nbeatsx the logic seems to imply that all the exogenous values must be present and filled in the forecast window.

# Test dataset and loader, to sample window with the day currently being predicted
# Test mask: 1s for 24 lead time
test_mask = np.zeros(len(Y_df_scaled))
test_mask[-offset:] = 1
test_mask[(len(Y_df_scaled) - offset + mc['output_size']):] = 0

assert test_mask.sum() == mc['output_size'], f'Sum of Test mask must be {mc["output_size"]} not {test_mask.sum()}'

ts_dataset_test = TimeSeriesDataset(Y_df=Y_df_scaled, X_df=X_df_scaled, ts_train_mask=test_mask)
test_ts_loader = TimeSeriesLoader(model='nbeats',
                                  ts_dataset=ts_dataset_test,
                                  window_sampling_limit=mc['window_sampling_limit_multiplier'] * mc['output_size'],
                                  offset=offset - mc['output_size'], # To bypass leakeage protection
                                  input_size=int(mc['input_size_multiplier'] * mc['output_size']),
                                  output_size=int(mc['output_size']),
                                  idx_to_sample_freq=24,
                                  batch_size=int(mc['batch_size']),
                                  is_train_loader=True,
                                  shuffle=False)

...

 _, y_hat_split, y_hat_decomposed_split, _ = model.predict(ts_loader=test_ts_loader,  return_decomposition=True)             

if mc['normalizer_y'] is not None:
    y_hat_split = scaler_y.inv_scale(x=y_hat_split)

print('Prediction: ', y_hat_split)

Am I misunderstanding something? Why is the test logic forecasting on the last 24 periods using the exogenous data that otherwise wouldn't be available at forecast time?

Thanks in advance for you assistance. This is a great project

datasets of nbeatsx can not download

Hi, there! Datasets of nbeatsx can not be downloaded.
The following URL is invalid.
SOURCE_URL = 'https://sandbox.zenodo.org/api/files/da5b2c6f-8418-4550-a7d0-7f2497b40f1b/'

Non-EPF Example

Thank you for this paper and the code. I am excited to try out the NBEATSx.

Would it be possible for you to share a minimal notebook that applies the method not to the price forecasting task but some more general time-series problem? I tried doing this but don't understand the specific hyperparameter settings from your nbeatsx_example.ipynb. In particular what idx_to_sample_freq specifies and how the lag dict works:

include_var_dict = {'y': [-8,-4,-3,-2],
                    'Exogenous1': [-8,-2,-1],
                    'Exogenous2': [-8,-2,-1],
                    'week_day': [-1]}

I prepared some code that downloads a dataset that is frequently used in time-series examples. It is the Jena Climate dataset recorded by the Max Planck Institute for Biogeochemistry. The dataset consists of 14 features such as temperature, pressure, humidity etc, recorded once per 10 minutes. Data is available for 8 years.
So it has a more unusual time structure but also a very simple task: predict the future temperature (with exogenous variables).
Example settings could be

window_sampling_limit=365*4*24*6, input_size=7*24*6, output_size=24*6

Please let me know if this would be possible for you; it would help me a lot in understanding NBEATSx.

---

# Code that downloads the Jena Climate dataset
from zipfile import ZipFile
from urllib.request import urlopen
from io import BytesIO
import pandas as pd

r = urlopen("https://storage.googleapis.com/tensorflow/tf-keras-datasets/jena_climate_2009_2016.csv.zip").read()
zip_data = ZipFile(BytesIO(r))
csv_path  = zip_data.open("jena_climate_2009_2016.csv")
df = pd.read_csv(csv_path, parse_dates = ["Date Time"])

Number of stacks confusion

Hi. Thanks for the amazing paper and architecture.

I was trying to find out how many stacks in a generic model you used and I noticed in your paper:

"The original NBEATS configuration includes only one
generic stack with dozens of blocks, while our proposed model includes both the generic and
exogenous stacks, with the order determined via data-driven hyperparameter tuning. We
refer to this configuration as the NBEATSx-G model."

That confused me, because in the original NBEATS paper in a table listing N-BEATS-G hyperparameters (Table 18.) authors claim to have used 30 stacks with 1 block in each. Did you assume they had mismatched the numbers? To me, 30 blocks and 1 stack seems to be more reasonable as well.

Can you confirm that you used just 2 stacks (one generic, one exogenous)? Additionally, do you think it would make sense to add more stacks (idk what configuration would be best)?

Regards

Missing import statements in nbeats.py

Hi, I was exploring the repository and noticed that the loss metrics import statement in nbeats.py was missing some of the import statements causing errors in line 361 to 367.
Current the statement reads as from src.utils.numpy.metrics import mae, pinball_loss but I'm assuming the statement should include mape, smape, rmse, and mse loss (from src.utils.numpy.metrics import mae, pinball_loss, mape, smape, rmse, mse).

Not a big problem tbh, just thought I would bring it up.

Great work by the way!

FCNN realization doesn't take exogenous variables as arguments

Hi,

In the paper in eq. (1) it is stated, that FCNN in each block is a function of output of previous block/stack (y^{back}) + exogenous variables. Whereas, I see at https://github.com/cchallu/nbeatsx/blob/main/src/nbeats/nbeats_model.py#L122 that there's only y^{back} passed to FCNN. My question is: why is that? My reasoning for passing insample exog. variables as well is that they could provide additional information on how to scale the C_{s,l} context vector later on.

Hyperparameter files

Hello Authors,

I'd be grateful if you would also upload the hyperparameters files for each dataset / case. I wanted to check the value of your validation MAE and also the hyperparameter values.

Thanks!

insample_mask and outsample_mask

What these two masks use for? When assigning the variables, the outsample_mask are all one, and the insample_mask are all one except the padding. But when getting batch from dataloader, the outsample_mask slice a length of past, and insample_mask slice a length of future. Is there any connections between them? And what the purpose of these masks. I am confused abouth this.

nothing

I am using nbeatsx_example.ipynb, and add some codes as follow:

result jpg
https://github.com/943fansi/GuideOfP/blob/main/1.jpg

print(y_true.shape, y_hat.shape, block.shape)
plt.plot(range(168, 336), y_true.flatten(), label='Price')
plt.plot(range(168, 336), y_hat.flatten(), linestyle='dashed', label='Forecast')
plt.axvline(168, color='black')
plt.legend()
plt.grid()
plt.xlabel('Hour')
plt.ylabel('Price')

plt.figure()
print(block.shape, len(block))
plt.plot(range(168, 336), block[:, 0, :].flatten(), linestyle='dashed', label='block Forecast')
plt.plot(range(168, 336), block[:, 1, :].flatten(), linestyle='dashed', label='block Forecast')
plt.show()```

Forecasting with long output dimension

Hello,

First of all thank you for your paper, I have tried N-beatx and it's performing super well.

The problem is that I need to forecast not only Day-Ahead but seven days (so 168 hours) . I tried to put my data in daily and did a 7 forecast horizon but its not good, do you have any advice to try for that problem please ? I would be very thanksful.

(By the way, my data is similar to yours in your n-beatsx paper.

Thank you for your time

[Question] dropout_prob_exogenous

The dropout_prob_exogenous seems to be unused

Any advice, please.

Hi. I read your paper very well.
I want to do two experiments.

1. Multivariate regression
   You said "The NBEATSx model offers a solution to the multivariate regression problem" in your paper.
   I will try with a multivariate datasets.

2. A probabilistic forecast, not a point forecasting
   I'm going to apply 'Quntile regression'.
   I wonder if it can be implemented without difficulty in the current architecture.
   For example, simple 'loss function, ..' modifications, etc.

Please let me know if you have any advice.
Thank you.

[Question] SELU weights and dropout

Hi,

My name is Pablo Navarro. Your team and I have already exchanged a few mails over the wonderful paper you've made. Thanks again for the contribution.

Now that the code is released, I have a couple question over the implementation of the SELU activation function.

Weight init

For SELU, you force lecun_normal which is in turn a pass on the init_weights() function:

def init_weights(module, initialization):
    if type(module) == t.nn.Linear:
        if initialization == 'orthogonal':
            t.nn.init.orthogonal_(module.weight)
        elif initialization == 'he_uniform':
            t.nn.init.kaiming_uniform_(module.weight)
        elif initialization == 'he_normal':
            t.nn.init.kaiming_normal_(module.weight)
        elif initialization == 'glorot_uniform':
            t.nn.init.xavier_uniform_(module.weight)
        elif initialization == 'glorot_normal':
            t.nn.init.xavier_normal_(module.weight)
        elif initialization == 'lecun_normal':
            pass
        else:
            assert 1<0, f'Initialization {initialization} not found'

How come the weights are initialized as lecun_normal simply by passing? On my machine, default PyTorch initializes weights uniformly, not normally.

DropOut on SELU

I believe that in order to make SELU useful, you need to use AlphaDropout() instead of regular DropOut() layers (PyTorch docs).

I can't find anything wrapping AlphaDropOut() in your code. Can you point me in the right direction or give the rationale behind it?

Cheers and keep up the good work!

Can NbeatsX be used for multiple-variate datasets?

Thank you for your paper!
Can NbeatsX be used for multiple-variate datasets (not single variate) ?
Thanks a lot!

n_val_weeks in the paper and code do not match

Hi,

Will you please clarify the value for n_val_weeks used to replicate your results? Is it 52, or 42? In the paper, it is mentioned as 42 in several places, but in the example on the main page on github (to replicate results of NP dataset), it is 52. Has the value 42 been used in some datasets, and 52 in some, maybe by mistake?

Thank you.

Number of stacks confusion

Hi. Thanks for the amazing paper and architecture.

I was trying to find out how many stacks in a generic model you used and I noticed in your paper:

"The original NBEATS configuration includes only one
generic stack with dozens of blocks, while our proposed model includes both the generic and
exogenous stacks, with the order determined via data-driven hyperparameter tuning. We
refer to this configuration as the NBEATSx-G model."

That confused me, because in the original NBEATS paper in a table listing N-BEATS-G hyperparameters (Table 18.) authors claim to have used 30 stacks with 1 block in each. Did you assume they had mismatched the numbers? To me, 30 blocks and 1 stack seems to be more reasonable as well.

Can you confirm that you used just 2 stacks (one generic, one exogenous)? Additionally, do you think it would make sense to add more stacks (idk what configuration would be best)?

Regards