theia-4869/bicross-old

BiCross (Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer).

BiCross - Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer

News

• [2022/10/24] We added more visualization results in another repository: BiCross-vis.

Architecture

For more details, please refer to our paper on Arxiv.

Result

Synthetic to Real

Extreme Weathers

Scene Changes

Usage

Train the model via BiCross

python train.py

• Training stages:
  1. Since the pretrained parameters of DPT are trained on the ImageNet, when you train from scratch, please first pretrain the model on the source RGB to adapt to the depth estimation task,, changing the stage option in the train_config.json to pretrain and training for about 30 epochs.
  2. After the pretrain stage, set stage in train_config.json to crossmodality and continue training for another 10 epochs from source RGB to source spike.
  3. Finally, set stage in train_config.json to crossdomain and then continue training for about 20 epochs from source spike to target spike.

Test the trained model

python test.py

Visualize the results

python visualize.py

You can modify the configs for different training and testing configurations.

Datasets

Detail

Download

Coming soon (in Google Drive) !

Demo

bicross-old's Issues

Question about the eval_mask and uncertainty_head

Hi, I really thankful for you to share the code. In trainer_crossdomain.py the train function has set the
eval_mask[:, int(0.40810811 * height):int(0.99189189 * height), int(0.03594771 * width):int(0.96405229 * width)] = 1 and I wonder how to get the float numbers? Also, I noticed that the structures of uncertainty_head and head are nearly the same. I don't understand the reason for this similarity. Looking forward to the reply and appreciate for the paper, your idea is inspiring. Best.

About dataset

Hi, nice work! Where can I download the dataset used in your paper?