chan8972/spike-flownet

spike-flownet's Issues

selection of sp_threshold=0.875.., Reason?

@chan8972
would like to know what is the reason for choosing sp_threshold=0.875 in the line. Kindly help

Spike-FlowNet/main_spike_flow_dt1.py

Line 249 in 1061c21

sp_threshold = 0.75

ImportWarning

hello, Thanks for your great job !

while I met this problem and wonder what and where "spatial_correlation_sampler import spatial_correlation_sample" is.

1.ImportWarning: failed to load custom correlation modulewhich isneeded for FlowNetC"which is needed for FlowNetC", ImportWarning)>

in addtion, other two problem arise
2.ImportWarning: failed to load custom correlation modulewhich isneeded for FlowNetC"which is needed for FlowNetC", ImportWarning)>

3.site-packages/torchvision/transforms/transforms.py:853: UserWarning: Argument intepolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum."Argument interpolation should be of type InterpolationMode instead of int."

AEE results on the outdoor1

Hello, thank you for sharing your code and great work.

I am trying to confirm the AEE results of the pretrained checkpoint_dt1.pth.tar on the outdoor1.

But I cannot get the same results as Table 1 in your paper although I did have the exactly same results on indoor1, indoor2, and indoor3. Here are my results.

Mean AEE: 2.78, sum AEE: 2531.94, Mean AEE_gt: 3.17, sum AEE_gt: 3958.44, mean %AEE: 0.93, # pts: 32.00

Should I change any parameters for outdoor1? It seems outdoor is different from indoor.
Thank you for your kind help in advance.

code for GT

Could you provide the code for generating GT? Thank you~

Problems in transformer

Hi~, When I was trying to run the main_spike_flow_dt1.py and main_spike_flow_dt4.py
There is a problem.

Traceback (most recent call last):
  File "main_spike_flow_dt1.py", line 550, in <module>
    main()
  File "main_spike_flow_dt1.py", line 526, in main
    train_loss = train(train_loader, model, optimizer, epoch, train_writer)
  File "main_spike_flow_dt1.py", line 253, in train
    for ww, data in enumerate(train_loader, 0):
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 582, in __next__
    return self._process_next_batch(batch)
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
IndexError: Traceback (most recent call last):
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 99, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "main_spike_flow_dt1.py", line 148, in __getitem__
    aaa[:, :, p] = self.transform(aa[:, :, p])
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 49, in __call__
    img = t(img)
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 110, in __call__
    return F.to_pil_image(pic, self.mode)
  File "/home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 115, in to_pil_image
    if npimg.shape[2] == 1:
IndexError: tuple index out of range

How should I fix it? Thanks :)

would you please upload the dataset into another drive, like MEGA?

Hi, good job on SNN based optical-flow estimation. But the google drive is not easy for researchers in China to use for the download of the dataset. So, would you please upload the dataset into another cloud drive, like MEGA? Thank you.

Execution Time Issue

Hi,
Thanks for sharing your code!

I'm trying to reproduce the test results on the outdoor day 1 dataset you're listing in your paper. However, the code needs approx. 5 seconds to test just one sample (this adds up to 15 hours for the whole outdoor day 1 dataset) I located the issue to be in the estimate_corresponding_gt_flow function, likely because the whole ground truth is moved to an array each time, which consumes a lot of CPU and is generally slow.
Did you experience a similar issue and could you somehow avoid it?

Thanks for your help!

Visualization

Why are the results of my visualization here all black

Pretrain Images

Hello, thank you for sharing your code and great work.

I am trying to view the results of the pretrained checkpoint_dt1.pth.tar and checkpoint_dt4.pth.tar model on the indoor_flying1

But it seems that the Spike Image and Predicted Flow Output visualizations are very sparse, even with the 4 frame model (results for checkpoint_dt4.pth.tar on indoor_flying1 shown below).

I am running with the existing parameters in the code:
batch_size_v = 4, sp_threshold = 0.5

Should any of the parameters be changed to get a denser spike input and thus denser predicted flow output?