Questions about train set and training

[JinRanYAO]

Questions about train set and training

[Kait0]

Traffic light differ between Town 01/ 02 and the rest of the towns.
If you only train on Town 03/04 you basically fail at red lights by design, you should at least use either Town 01 or 02.

What does bad behavior mean?

1. The lowest batch sizes I have used were ~40 and that did not seem to cause problems. You training set is definitely too small you should train on all towns. Why are you not using a higher batch size if you have 4 GPUs? Or do you mean batch size of 16 per GPU?

2. You don't need more epochs the 40 you have here is enough. I would expect more data to help unless there is another problem here.

3. If you want to reproduce the results in the paper you need to train on all of the dataset.

4. Yes the pre-trained model is trained with the entire dataset. There is not more data.

Besides open loop losses are not really indicative on closed-loop performance.

[JinRanYAO]

Hello, thank you for your reply!

Traffic light differ between Town 01/ 02 and the rest of the towns. If you only train on Town 03/04 you basically fail at red lights by design, you should at least use either Town 01 or 02.

What does bad behavior mean?

It doesn't just fail the red light test, it also fails to maintain accurate lanes and does not drive in the middle of the lane, does not turn into the lane accurately when turning at an intersection, causing collisions with other vehicles.

1. The lowest batch sizes I have used were ~40 and that did not seem to cause problems. You training set is definitely too small you should train on all towns. Why are you not using a higher batch size if you have 4 GPUs? Or do you mean batch size of 16 per GPU?

I'm sorry I didn't explain this clearly. I have a batch size of 16 for each GPU.

2. You don't need more epochs the 40 you have here is enough. I would expect more data to help unless there is another problem here.
3. If you want to reproduce the results in the paper you need to train on all of the dataset.
4. Yes the pre-trained model is trained with the entire dataset. There is not more data.

Thank you for your advice, and I will try to use more data for training.

Besides open loop losses are not really indicative on closed-loop performance.

Do you think the abnormal behavior in CARLA I just mentioned are also caused by insufficient training data? Or could there be other errors?

[Kait0]

Do you think the abnormal behavior in CARLA I just mentioned are also caused by insufficient training data? Or could there be other errors?

Hard to tell but low amounts of data could cause these issues, yes.

[JinRanYAO]

OK, I will try to use the whole dataset to train. Thank you very much!