Comments (19)
Hi,
I had a similar situation with two machines, one had gtx1080ti installed and one with titanV installed. The gtx1080ti produced better results than the titanV one. Not sure why! I noticed overall even with the same code different machines with different versions of software stack produce different results. How far is your difference?
from social-stgcnn.
Hi,
I had a similar situation with two machines, one had gtx1080ti installed and one with titanV installed. The gtx1080ti produced better results than the titanV one. Not sure why! I noticed overall even with the same code different machines with different versions of software stack produce different results. How far is your difference?
Hi,
Thank you very much for your reply.
Generally, for ADE, the difference is around 3%, while for FDE the difference is more than 5%. The max difference I got for FDE was 9%! For instance, last week I got an FDE 0.69, but with the same code it is 0.78 now...
That's too strange and I don't why this happen...I am not sure what I have installed over the weekend and why these affect the results so much. I have tried to setup several env with different version of python and torch, but still I cannot reproduce the one with FDE 0.69...
from social-stgcnn.
By the way, using your code (the exact CVPR version), now after training with my machine, the ADE is 0.466 while FDE is 0.802, which are much worse than you reported in the paper. Previously I can obtain better results, but now I cannot reproduce them...
from social-stgcnn.
I think it's highly related to the environment + random seeds maybe? The exact CVPR version have the weights which the results were reported which can be helpful. This is the requirements file at the time of CVPR https://github.com/abduallahmohamed/Social-STGCNN/blob/master/checkpoint/requirements.txt I just uploaded it, it might be helpful to you.
from social-stgcnn.
I think it's highly related to the environment + random seeds maybe? The exact CVPR version have the weights which the results were reported which can be helpful. This is the requirements file at the time of CVPR https://github.com/abduallahmohamed/Social-STGCNN/blob/master/checkpoint/requirements.txt I just uploaded it, it might be helpful to you.
Hi,
Thank you for your file. To be honest I don't know the reason. Possibly it is related to the environment. I will look into your requirements file maybe I can find some differences.
from social-stgcnn.
I think it's highly related to the environment + random seeds maybe? The exact CVPR version have the weights which the results were reported which can be helpful. This is the requirements file at the time of CVPR https://github.com/abduallahmohamed/Social-STGCNN/blob/master/checkpoint/requirements.txt I just uploaded it, it might be helpful to you.
Hi,
I found that when I run train.sh, the data processing stage showed this. Is this normal? I mean some progress bar wer not 100%. When you trained your model, how was this shown?
Thanks!
8%|████▊ | 165/2076 [00:04<00:54, 34.93it/s]Processing Data .....
9%|█████▊ | 197/2076 [00:05<00:59, 31.53it/s]Processing Data .....
0%|▏ | 6/2112 [00:01<06:53, 5.10it/s]Processing Data .....
0%| | 3/2322 [00:00<06:03, 6.38it/s]Processing Data .....
100%|████████████████████████████████████████████████████████████| 2076/2076 [01:19<00:00, 26.10it/s]
14%|████████▋ | 300/2112 [01:16<09:20, 3.23it/s]Processing Data .....
100%|██████████████████████████████████████████████████████████████| 530/530 [00:26<00:00, 19.77it/s]
27%|████████████████▎ | 621/2322 [01:43<01:13, 23.15it/s] Data and model loaded
from social-stgcnn.
from social-stgcnn.
Hi, This is normal it’s a progress bar issue when the screen is tight. Also you get multiple progress bars(5) per each training session.
On Wed, Oct 21, 2020 at 4:20 PM Xingchen Zhang @.***> wrote: I think it's highly related to the environment + random seeds maybe? The exact CVPR version have the weights which the results were reported which can be helpful. This is the requirements file at the time of CVPR https://github.com/abduallahmohamed/Social-STGCNN/blob/master/checkpoint/requirements.txt I just uploaded it, it might be helpful to you. Hi, I found that when I run train.sh, the data processing stage showed this. Is this normal? I mean some progress bar wer not 100%. When you trained your model, how was this shown? Thanks! 8%|████▊ | 165/2076 [00:04<00:54, 34.93it/s]Processing Data ..... 9%|█████▊ | 197/2076 [00:05<00:59, 31.53it/s]Processing Data ..... 0%|▏ | 6/2112 [00:01<06:53, 5.10it/s]Processing Data ..... 0%| | 3/2322 [00:00<06:03, 6.38it/s]Processing Data ..... 100%|████████████████████████████████████████████████████████████| 2076/2076 [01:19<00:00, 26.10it/s] 14%|████████▋ | 300/2112 [01:16<09:20, 3.23it/s]Processing Data ..... 100%|██████████████████████████████████████████████████████████████| 530/530 [00:26<00:00, 19.77it/s] 27%|████████████████▎ | 621/2322 [01:43<01:13, 23.15it/s] Data and model loaded — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFIVLG7OYHJU6NR6J232UBLSL5GDVANCNFSM4SZ7LPNQ .
-- ------------------------------------------ Abduallah Mohamed abduallahmohamed.com
Many thanks for your reply!
I have tried your requirements file, and also tried to recover my previous running environment. But so far, the results are still not good.
Apart from the running environment, you mentioned that the random seeds may affect the results. But I think this only affect the testing results, and will not affect training, right?
Thanks for your help!
from social-stgcnn.
from social-stgcnn.
I think even though the environment might be the same a different GPU architecture and Cuda version can effect it too. Not sure about the random seed, nonetheless you have the weights in the repo to reproduce the results. On Fri, Oct 23, 2020 at 9:37 AM Xingchen Zhang [email protected] wrote:
Hi, This is normal it’s a progress bar issue when the screen is tight. Also you get multiple progress bars(5) per each training session. On Wed, Oct 21, 2020 at 4:20 PM Xingchen Zhang @.***> wrote: I think it's highly related to the environment + random seeds maybe? The exact CVPR version have the weights which the results were reported which can be helpful. This is the requirements file at the time of CVPR https://github.com/abduallahmohamed/Social-STGCNN/blob/master/checkpoint/requirements.txt I just uploaded it, it might be helpful to you. Hi, I found that when I run train.sh, the data processing stage showed this. Is this normal? I mean some progress bar wer not 100%. When you trained your model, how was this shown? Thanks! 8%|████▊ | 165/2076 [00:04<00:54, 34.93it/s]Processing Data ..... 9%|█████▊ | 197/2076 [00:05<00:59, 31.53it/s]Processing Data ..... 0%|▏ | 6/2112 [00:01<06:53, 5.10it/s]Processing Data ..... 0%| | 3/2322 [00:00<06:03, 6.38it/s]Processing Data ..... 100%|████████████████████████████████████████████████████████████| 2076/2076 [01:19<00:00, 26.10it/s] 14%|████████▋ | 300/2112 [01:16<09:20, 3.23it/s]Processing Data ..... 100%|██████████████████████████████████████████████████████████████| 530/530 [00:26<00:00, 19.77it/s] 27%|████████████████▎ | 621/2322 [01:43<01:13, 23.15it/s] Data and model loaded — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment) <#34 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFIVLG7OYHJU6NR6J232UBLSL5GDVANCNFSM4SZ7LPNQ . -- ------------------------------------------ Abduallah Mohamed abduallahmohamed.com Many thanks for your reply! I have tried your requirements file, and also tried to recover my previous running environment. But so far, the results are still not good. Apart from the running environment, you mentioned that the random seeds may affect the results. But I think this only affect the testing results, and will not affect training, right? Thanks for your help! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFIVLG6CY5QK3UOY47LJEP3SMGIIZANCNFSM4SZ7LPNQ .
-- ------------------------------------------ Abduallah Mohamed abduallahmohamed.com
Yes, I can reproduce your results using your pretrained model. But I am just wondering why the performance are so easily affected.
Anyway, many thanks for your help!
from social-stgcnn.
from social-stgcnn.
Hi,
In your paper you mentioned 'A property to note regards TXP-CNN layer that it is not a permutation invariant as changes in the graph embedding right before TXP-CNN leads to different results.' Is this the reason why results are different? Because in each run, the input is actually different, for instance the order of pedestrians and sequences. Do you think this is one reason?
from social-stgcnn.
Hi,
I don't think so. The input arrangement of the pedestrians will results in the same graph embedding (permuted), but permuting the embeddings will result in a different behavior. Nonetheless,
in the paper:
The 100% is the full data, we show the mean and std of our results based on different random seeds. On each run on a machine you will start the weights with a different set of variables that will lead to a different minima. I hope this helps!
from social-stgcnn.
Thank you very much for your reply!
from social-stgcnn.
In your script, you used train.sh to train five models together. Can I train five models one by one using the same hyper-parameters? I mean is this allowed for fair comparison?
Also, do we have to use the same hyper-parameters (the same learning rate, batch size, etc.) to train five models right? can we fine-tune each model (ETH, HOTEL, UNIV, ZARA1, ZARA2) and computer the average ADE and FDE?
from social-stgcnn.
from social-stgcnn.
Hi Abduallah,
If I understand you correctly, it is allowed to fine tune each model (for example using different hyper-parameters for each model) as long as I pick up the best model based on the validation set and report the results on the test set. However, people normally use the same hyper-parameters, right? Because in your paper and other related papers, I just saw you guys reported one set of hyper-parameters. I assumed that these are applied to all five models.
Thank you very much for your reply! You are very kind!
Xingchen
from social-stgcnn.
from social-stgcnn.
Hi Abduallah,
Thank you very much for your reply!
Bests,
Xingchen
from social-stgcnn.
Related Issues (20)
- Why should we choose the normalized laplacian matrix? HOT 1
- about torch HOT 2
- anorm problem about step_rel and step_ HOT 4
- Data visualization HOT 3
- Why does the loss value change from positive to negative HOT 1
- PROBLEM About TCN HOT 1
- could min-pedestrian greater than 1? HOT 1
- How to handle trajectories with missing frames? HOT 2
- Visualize trajectory in picture HOT 6
- Details about data processing HOT 3
- Questions about data process
- I am very curious about the element in your proposed adjacency matrix
- I am very curious about the element in your proposed adjacency matrix HOT 3
- When I test my own model, some datasets work fine, but the univ dataset reports errors. The specific error message is as follows. HOT 1
- details about dataset .txt file encoding type. HOT 1
- A question of determining linear or nonlinear
- During visualization, the jupyter notebook encountered the following error
- Details about Gaussian path visualization
- Question about the code in basic gcn unit
- final predicted trajectory data
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from social-stgcnn.