tsantunes/mlrestaurantsalesforecasting

Python code for the generation and testing of machine learning models, including the Temporal Fusion Transformer recurrent neural network model. Several Jupyter Notebook files are available with different test suites to cover feature testing, model tuning, one-day forecast tests, and one-week forecast tests. Also included are three datasets.

Machine Learning Restaurant Sales Forecasting

Python code for the generation and testing of machine learning models, including the Temporal Fusion Transformer recurrent neural network model. Several Jupyter Notebook files are available with different test suites to cover feature testing, model tuning, one-day forecast tests, and one-week forecast tests. Also included are three datasets. An actual sales dataset and two derivative datasets made from daily and weekly sales differencing.

To render the Jupyter Notebook pages reliably without downloading, copy and paste the URL for this repository at: https://nbviewer.jupyter.org/ The files will be rendered for quick view.

The supplied dataset is organized as follows:

There are two files holding data.

(1) RestaurantDataVets_All_2to5.csv

The raw input data has been preprocessed to have many features with the target being '2to5' which is the amount of sales generated between 2:00 and 5:59 PM on that day. The Jupyter Notebook code handles the addition of lookback features, one-hot encoding, and data scaling. The first 7 days are used to generate statistical measurements, and must be thrown out before training.

(2) RestaurantDataVets_All_2to5_Differenced.csv

The differenced dataset has additional features and targets to benefit from. The targets are: '2to5', 'DailyDifference', 'WeeklyDifference', and 'DiffDifference', and care should be made to drop targets not being used before training. While '2to5' are actual sales, daily difference is the difference in sales between today and yesterday, weekly difference is the difference between today and last week, and diff difference applies both differencing techniques. The Jupyter Notebook code handles the addition of lookback features, one-hot encoding, and data scaling. The first 14 days are used to generate statistical measurements, and must be thrown out before training.

The raw dataset may be acquired by directly reaching out to the author at [email protected].

The Jupyter Notebook files are organized as follows:

There are 5 Jupyter Notebook files holding training/testing procedures for a large survey of models. Files should be run in cell order to ensure desired outputs.

(1) Scikit Feature Selection.ipynb

A feature test is conducted on all surveyed non-recurrent models. There are taken directly from the scikit-learn library, and have few changes made to hyperparameters. Features are first ranked and then cross-validation is used to determine the MAE score generated from the model using an iteratively increasing number of features from n=1 to n='all'. First feature testing for the Actual, Daily Differenced, and Weekly Differenced datasets are completed with one-day forecasting. The MAE scores are recorded and then outputted to an .csv file for analysis. The process is then repeated for one-week forecasting using a model wrapper to use one instance for a full week of prediction.

(2) Keras Feature Selection.ipynb

A feature test is conducted on all surveyed recurrent models. There are taken directly from the keras library, and have few changes made to hyperparameters. Early stopping and dropout layers aid in prevention of overfitting. Features are first ranked and then a train/test split is used to determine the MAE score generated from the model using an iteratively increasing number of features from n=1 to n='all'. First feature testing for the Actual, Daily Differenced, and Weekly Differenced datasets are completed with one-day forecasting. The MAE scores are recorded and then outputted to an .csv file for analysis. The process is then repeated for one-week forecasting increasing the number of output nodes to use one instance for a full week of prediction.

(3) TransformerTuning.ipynb

A feature test is not included for the TFT model, instead we estimate the best number of features to use from the LSTM model which TFT implements as it's recurrent backbone. TFT is so much larger than the other models that tuning is a necessity. In this file we tune hyperparameters for the TFT model using all three datasets as used in the feature test. Also, for each dataset we tune twice. Once with the full feature selection and once with the best-estimated number for a total of six TFT models tuned for one-day forecasting. The process is again repeated for one-week testing.

Careful: This tuning process can take 8+ hours per test.

(4) FinalTestingOneDay.ipynb

The best models, number of features, and tuning results are taken and applied to all three datasets for one-day forecasting. Each model will output its forecast over the same predefined test dataset so direct comparisons may be made. Non-recurrent models are tested first, then recurrent models, then TFT is shown in its own section. The author has already set up the optimal number of features, so we may run this sequentially to see interesting results.

(5) FinalTestingOneWeek.ipynb

The best models, number of features, and tuning results are taken and applied to all three datasets for one-week forecasting. Each model will output its forecast over the same predefined test dataset so direct comparisons may be made. The test for one-week forecasting includes starting on each day of the week so a sliding test window is used, and each forecast is displayed. The MAE of each forecast is recorded, and the mean MAE and standard deviation is calculated to determine model stability. Non-recurrent models are tested first, then recurrent models, then TFT is shown in its own section. The author has already set up the optimal number of features, so we may run this sequentially to see interesting results.

Note: The following features were removed from one-week forecasting: DailyAvg, DailyBusyness, and AvgDailyDiff due to concerns of improper information gain, however this was a mistake, and these features really can be used. Luckily, they rank low on feature importance, so it is likely this hurt performance very little.

Workflow:

In Figure 1 we display the general workflow used to create a model.

Figure 1: Model Training/Testing Flowchart. We display the lifecycle from raw preprocessed data to fully functional forecasting models. Collected raw data is organized as described in the text. Data scaling brings all features in line with one another. The feature test provides the optimal number of features. The best non-recurrent and all recurrent models move on for final testing. Beyond the increased class-size for one-week forecasting, the flowchart methodology is exactly the same. The testing files can be run sequentially as listed above from 1-5. The optimal number of features and tuning results need to be added manually to the final testing suits, but this has already happened, so the test suits can just be run to see the completed results.