nachiket92/conv-social-pooling

Code for model proposed in: Nachiket Deo and Mohan M. Trivedi,"Convolutional Social Pooling for Vehicle Trajectory Prediction." CVPRW, 2018

Convolutional Social Pooling

Code for model proposed in [1]. Implemented in Pytorch 0.4.0

[1] Nachiket Deo and Mohan M. Trivedi,"Convolutional Social Pooling for Vehicle Trajectory Prediction." CVPRW, 2018

conv-social-pooling's Issues

How do you calculate the NLL loss？

Could you tell me where does the formula come from?

How long do you run this code(preprocess_data.m)?

Could you add some documentation?

Nothing too extensive, just basic instructions/commands on how to use your code? Maybe a short demo?

Thanks!

Use cpu instead of gpu

I don't have NVIDIA graphic installed on my system. I'm wondering if I can run this code with a limited dataset on CPU instead of GPU?

Dataset

Hey thanks for sharing the code. Any plan on releasing the preprocessed dataset. The .mat files?

Future trajectory encoding

What kind of future traj do we need for evaluation? ego-vehicle or surrounding vehicle?

utils.py line 80:
def getFuture(self, vehId, t,dsId): vehTrack = self.T[dsId-1][vehId-1].transpose() refPos = vehTrack[np.where(vehTrack[:, 0] == t)][0, 1:3] stpt = np.argwhere(vehTrack[:, 0] == t).item() + self.d_s enpt = np.minimum(len(vehTrack), np.argwhere(vehTrack[:, 0] == t).item() + self.t_f + 1) fut = vehTrack[stpt:enpt:self.d_s,1:3]-refPos return fut

Maneuver output

Hi ,
I want know what the relationship between output channels of lat_pred and lon_pred and the maneuver, I intend to use this maneuver to get the most possible predict trajectory.
waiting for your answer .

Why use ''mashed_scatter_" before conv social pooling in the model?

1. Why use ''mashed_scatter_" before conv social pooling in the model? How to get mash from data?
2. The neighbor's pass position data is converted a tensor('nbrs') input to the model? But in the paper, every neighbor's position is inputted a corresponding LSTM.

Training problem：Floating point exception (core dumped)，when I change "shuffle=True" to "shuffle=False"

Hi，I ran your code with GPU，cuda10.0，pytorch1.2.0. How can I copy with this problem？

what is the meaning of the function logsumexp

Why is the difference required after the maximum value is taken

the code is following

## Helper function for log sum exp calculation:
def logsumexp(inputs, dim=None, keepdim=False):
    if dim is None:
        inputs = inputs.view(-1)
        dim = 0
    s, _ = torch.max(inputs, dim=dim, keepdim=True)
    outputs = s + (inputs - s).exp().sum(dim=dim, keepdim=True).log()
    if not keepdim:
        outputs = outputs.squeeze(dim)
    return outputs

what is the meaning of nbrs?

hi, when I see your code, I don't understand the meaning of nbrs. Could you explain this variable's mean?

Can you provide the pretrained models?

Can you provide the pre-trained LSTM model? Thanks !

nll

Why my training results differ greatly from the results of the original paper, mainly nll. What's wrong with rmse

preprocess_data.m

The program with 6 workers has been running for 40 hours and still hasn't finished.
What's wrong?

Considering seq2seq model

First of all, congratulations for this fantastic work!

Regarding the encoder-decoder architecture, have you ever considered using a seqquence-2-sequence model? In a seq2seq model, the decoder's input at t=n is the output of the decoder itself at t=n-1. I don't know whether it would be of any benefit, and it complicates the training and inference a little bit (typically, teacher forcing mode is used)

FileNotFoundError: [Errno 2] No such file or directory: 'data/TrainSet.mat'

Hello,
could you also please upload the dependent files,
Thanks in advance!

Error log

/home/npv0/exp/pytorch/venvpytorch/bin/python /home/npv0/exp/conv-social-pooling/train.py
Traceback (most recent call last):
File "/home/npv0/exp/pytorch/venvpytorch/lib/python3.5/site-packages/scipy/io/matlab/mio.py", line 33, in _open_file
return open(file_like, 'rb'), True
FileNotFoundError: [Errno 2] No such file or directory: 'data/TrainSet.mat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/npv0/exp/conv-social-pooling/train.py", line 44, in
trSet = ngsimDataset('data/TrainSet.mat')
File "/home/npv0/exp/conv-social-pooling/utils.py", line 14, in init
self.D = scp.loadmat(mat_file)['traj']
File "/home/npv0/exp/pytorch/venvpytorch/lib/python3.5/site-packages/scipy/io/matlab/mio.py", line 141, in loadmat
MR, file_opened = mat_reader_factory(file_name, appendmat, **kwargs)
File "/home/npv0/exp/pytorch/venvpytorch/lib/python3.5/site-packages/scipy/io/matlab/mio.py", line 64, in mat_reader_factory
byte_stream, file_opened = _open_file(file_name, appendmat)
File "/home/npv0/exp/pytorch/venvpytorch/lib/python3.5/site-packages/scipy/io/matlab/mio.py", line 39, in _open_file
return open(file_like, 'rb'), True
FileNotFoundError: [Errno 2] No such file or directory: 'data/TrainSet.mat'

plot the heat map

how to plot the heat map just like the Figure.5 in your paper?
is there any demo or tutorial?
thanks.

baseline not included

I was wondering if you could include the code for the CV Kalman filter as well that you used in your results table?

HighD data process file

Dataset Problem

First of all, this is a fantastic work！
I have downloaded ‘Next_Generation_Simulation__NGSIM__Vehicle_Trajectories_and_Supporting_Data.csv’ dataset， but it can't match your code. I heard that the dataset had been updated.
So, could you please give us a dataset link which includes 6 txt? Or just give us your dataset detail format. I know you mentioned some in preprocess_data.m, but that is too rough. Or just upload example txt, please!

Result of the evaluate.py

Thanks for your contribution.
I have a question about the result of evaluate.py.
Why do I get a tensor after I run the evaluate.py?
And my result is as follows.
In my opinion, the result should be the values of loss and accuracy as well as the results of train.py.

Meaning of fut_pred in details

The decode() function, in the highwayNet class, is the one returning the future trajectory predicted. Before returning the trajectory, fut_pred is modified by outputActivation() in utils.py.
The return value of outputActivation() is a concatenation of muX, muY, sigX, sigY and rho. The comment above tells us this output layer comes from Graves, 2015 but I don't understand the meaning of these 5 variables.

Can you explain to me the meaning of these variables?

NLL loss has negative values

When evaluating my trained model with NLL as metric, I realize negative values for early timesteps (such as -3.27 at 1 second). This would my predicted probability for the gt trajectory point is in the area of 1000 instead of a maximum of 1. I trained the model on a custom dataset and not on NGSIM. Any ideas where this comes from?

File Not present

Hi,

I have downloaded all the zip files as well as the large csv file "Next_Generation_Simulation__NGSIM__Vehicle_Trajectories_and_Supporting_Data.csv" , but i couldnt find few parameters such as {DatasetID, Lateral maneuver, Longitudinal maneuver}. May i know where i can find those data fields?

As mentioned in the "preprocess_data.m" the below files are not present in the zip files which i have downloaded.
us101_1 = 'raw/us101-0750-0805.txt';
us101_2 = 'raw/us101-0805-0820.txt';
us101_3 = 'raw/us101-0820-0835.txt';
i80_1 = 'raw/i80-1600-1615.txt';
i80_2 = 'raw/i80-1700-1715.txt';
i80_3 = 'raw/i80-1715-1730.txt';

Kindly help.

HighD data process file

Could you send me the matlab file for processing highD or processed dataset? my email: [email protected]

About output

Could you tell me if your output consists of future X and Y co-ordinates of predicted vehicles or not and if yes where is it?
I've taught it is in fut-pred vector but it seems not.

evaluate.py

Hi,
In the 'evaluate' program, what are the meanings of the two parameters, 'in_length = 16' and 'out_length = 25'? Don't we want to predict the vehicle trajectory in the next 5s? Why 'out'_length = 25?

How to get the raw input files?

Hi, I couldn't find the below files from NGSIM website. Could you provide me the link for those data? Thanks!

us101_1 = 'raw/us101-0750-0805.txt';
us101_2 = 'raw/us101-0805-0820.txt';
us101_3 = 'raw/us101-0820-0835.txt';
i80_1 = 'raw/i80-1600-1615.txt';
i80_2 = 'raw/i80-1700-1715.txt';
i80_3 = 'raw/i80-1715-1730.txt';

Covariance matrix not positive semidefinite

When I try to compute the explicit probabilities of the bivariate gaussian distribution, I first define the covariance matrix as [[1/sigX, rho], [rho, 1/sigY]]. This is working for most of the predictions. However, sometimes this matrix is not positive semidefinite which throws an error. Any clue on what am I doing wrong?

Could you provide a direct link of the dataset used in this paper? thanks~

Inconsistency between X and Y loss

utils lines 170, 172 (for feet, meters respectively) have
0.5*torch.pow(sigY, 2)*torch.pow(y-muY, 2) for Y
but
torch.pow(sigX, 2) * torch.pow(x - muX, 2) for X

Change batch size from 128 to 1 get "Process finished with exit code 136 (interrupted by signal 8: SIGFPE)"

Before running evaluate.py, I change the input data's batch size from 128 to 1. After running it, I get "Process finished with exit code 136 (interrupted by signal 8: SIGFPE)". I find some information from the internet, somewhere has float point errors, I doubt it happens in the script of model.py. So how can I solve this problem?

What is the meaning of op_mask in the code?

Hello, when I see your code, I don't understand the meaning of op_mask. Could you explain this variable's mean?

Text files used in preprocessing not provided

@nachiket92, the script you provided uses preprocessed text files to generate the .mat files. Could you provide code to generate those text files? Or even the preprocessed text files should be fine.

outputs of train.py

Standard deviation in prediction decreases for points further in the future

The figure below shows the bivariant gauss distribution of the model's prediction. It contains the 25 timesteps of the most likely maneuver (fut_pred_max). The model was trained with default settings on the NGSIM data set as described.
As you see in the figure, predicted points that are nearer in the future have a higher variance than points that are further in the future. Looking at the pure number confirms this.

How is this possible? Usually, the further the prediction is in the future the less certain you are about the exact position. So the variance should increase over time. Any help is appreciated!

Covariance matrix of the bi-variate Gaussians

It seems that the covariance matrix of the bi-variate Gaussian is assumed to be diagonal, isn't it? isn't it a strong assumption?

The seq2seq model function

Hi,
How are the encoder and decoder functions defined, and how are the hidden states of each time step transformed into background variables.