About the input sequence and PID controller

[autonomousvision]

About the input sequence and PID controller

[ap229997]

1. seq_len = 1 means we only consider input from a single timestep. Since the transformer takes a sequence of tokens as input, we convert the feature map that we get from the convolutional encoder into a 1D sequence of tokens. For example, 8x8 feature map is converted into 64 tokens, so these 64 tokens form a sequence.
2. Predicting future entities, in general, helps the model to implicitly learn anticipation/forecasting capabilities which is desirable. We experimented with predicting multiple timesteps into the future and 4 timesteps worked the best. Another use of the 4 waypoints could be to design a better controller, e.g. fitting a curve over the 4 waypoints rather than using just a few waypoints.

[oskarnatan]

Ok, got it. Thanks!