Missing/incorrect files for preprocessing

[aws-samples]

Missing/incorrect files for preprocessing

[oentaryorj]

It seems that we also need to manually create data and distance_matrix subfolders. Would be good to handle this in preprocessing.py - e.g., https://www.geeksforgeeks.org/how-to-create-directory-if-it-does-not-exist-using-python/

Nevertheless, after doing the above, I still see this error:

Traceback (most recent call last):
  File "preprocessing.py", line 459, in <module>
    raise Exception('Missing the name of the parquet file (e.g. lmc_route_full_1637316909.parquet) generated by the gen_route action')

[chenwuperth]

Hi @oentaryorj , thank you very much for reporting the issue. Can you please git pull the latest from the main branch?

I have fixed the file naming issues in the code, so depends on the --mode argument, it will change the file name (in fact sub-directory name) accordingly.
I have also added the auto-creation of sub-folder distance_matrix and zone_list. The sub-folder data is not auto-created only because it is possible the this folder is outside of the git repository (depends where the dataset is downloaded to). So you could use the argument --data_dir to tell the preprocessing script where the train or eval dataset is
The second error was due to the missing parameter --pkfn, and I have added that in the README.md instruction. This file is only generated after you run the python preprocessing.py --act gen_route
Please let me know if you see any other problems. Thanks again for your help.