Question about result table on paper

[autonomousvision]

Question about result table on paper

[ap229997]

1. The variance for transfuser and other models comes from two sources - training seed and evaluation variance. Training seed generally affects the initialization of the network and the order of the training data. Evaluation variance is due to the stochasticity in the behavior or other dynamic agents and traffic lights in the scene.

2. In our experiments, we found that the traffic manager in CARLA 0.9.10 still led to some evaluation variance. In the newer CARLA version, the traffic manager is supposed to be deterministic so that should reduce the variance but I have not experimented with the newer versions yet.

[Kin-Zhang]

• The variance for transfuser and other models comes from two sources - training seed and evaluation variance. Training seed generally affects the initialization of the network and the order of the training data. Evaluation variance is due to the stochasticity in the behavior or other dynamic agents and traffic lights in the scene.
• In our experiments, we found that the traffic manager in CARLA 0.9.10 still led to some evaluation variance. In the newer CARLA version, the traffic manager is supposed to be deterministic so that should reduce the variance but I have not experimented with the newer versions yet.

Thanks for replying, for first one yes the variance is from seed and initial But according to the result table and variance maybe shows that the Geometric method get higher scores since low variance here.

And for Carla, I check on 0.9.10 the documents didn’t have deterministic method but the leaderboard based on Carla 0.9.10.1 which use the getter method on code, I run several time on my computer find that with 0.9.10.1, it’s true for deterministic.

Finally Thanks for your reply, all the issue part helps me learn more about your method!