aws-samples/amazon-sagemaker-amazon-routing-challenge-sol

AWS Last Mile Route Sequence Optimization

amazon-sagemaker-amazon-routing-challenge-sol's Issues

Missing/incorrect files for preprocessing

Hi,

Thanks for this great repository.

I downloaded the AWS Last-Mile Challenge dataset (step 2; via aws s3 sync --no-sign-request s3://amazon-last-mile-challenges/almrrc2021/ ./data/), and am now trying to preprocess the data (step 3). However, it seems that some file names are missing/inconsistent when I tried to run the script below:

train_data_dir=almrrc2021-data-training
eval_data_dir=almrrc2021-data-evaluation

# generate package information in Parquet (training)
python preprocessing.py --act gen_route --data_dir  data/${train_data_dir}
# generate travel time matrix for all stops in a route (training)
python preprocessing.py --act gen_dist_mat --data_dir  data/${train_data_dir}
# generate zone information for each stop (training)
python preprocessing.py --act gen_zone_list --data_dir  data/${train_data_dir}
# generate ground-truth zone sequence for each route
python preprocessing.py --act gen_actual_zone --data_dir  data/${train_data_dir}
# generate package information in Parquet (evaluation)
python preprocessing.py --act gen_route --data_dir  data/${eval_data_dir}
# generate travel time matrix for all stops in a route (evaluation)
python preprocessing.py --act gen_dist_mat --data_dir  data/${eval_data_dir}
# generate zone information for each stop (evaluation)
python preprocessing.py --act gen_zone_list --data_dir  data/${eval_data_dir}

I noticed that the JSON files under almrrc2021-data-training folder are slightly different from those under almrrc2021-data-evaluation, e.g., almrrc2021-data-training/model_score_inputs/new_actual_sequences.json vs. almrrc2021-data-evaluation/model_score_inputs/eval_actual_sequences.json.

Would it be appropriate to rename the former to eval_actual_sequences.json in this case? Or is it better to handle this in preprocessing.py, e.g., the LMC_ACTUAL_FN variable?