raykoooo/iast

IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

iast's Issues

cannot reproduce the result

Hi
I try to implement the release code on T4 for GTA5->Cityscapes and found the best result I can get is 50.7, which happens in the sl_1 and I saw overfitting problem and finally get 49.5 for sl_3. I wonder whether you have similar results and only report the best one during all the training process.

Thanks

About model selection

IAST/scripts/self_training_only/run_gtav2cityscapes_self_traing_only_t4.sh

Line 7 in a10bee6

python main.py --config_file config/gtav2cityscapes_t4/run_task/sl_2.yaml --resume_from ${BASE_WORK_DIR}/sl_1/epoch_2.pth --pseudo_resume_from ${BASE_WORK_DIR}/sl_1/epoch_1.pth --work_dir ${BASE_WORK_DIR}/sl_2

Thanks for your good work and the code. In the configuration files, is there any special reason to use the model after first epoch for the pseudo label generation but resume from the last model for the next round?

Problem

Hello, thank you very much for your open source code.
Because of the limited capacity of the GPU, I can only use the weights that you trained with the graphics card T4.
I ran the GTA5 dataset and it was similar to yours.
This is because GitHub has not yet updated the weight of T4 for the Synthia dataset. So I trained this part from scratch.When I started training from scratch, I found that the results were a little poor.I don't understand this question, so I want to consult you.I would also like to ask when you can update the training weights for the Synthia dataset.
Thank you very much.

missing files *labelTrainIds.png

Hi
Thanks for the good work. When I try to use the code, I got the missing file error that shows all the file with name *labelTrainIds.png from the cityscape dataset are missing. The ground truth files I can find have name without 'Train'. Could you help me check whether they are the same files or not.

If the files are the same, when I try to use your code I find after training the model with 800 iterations, I can only get mIOU 0.0049, which seems to be something wrong. I just use the file 'run_gtav2cityscapes_self_traing_only_t4.sh' with the pertained model you released.

Thanks
Yunsheng

Are the miou results described in the paper a single stage result? (synthia->cityscapes)

Hello, thanks for your amazing works.

I wonder if the miou result of your method for synthia->cityscapes (Table 7 in your paper) is a single stage result or multi-stage result.
If the performance is result of single stage, can you tell the miou result when performing multi-stage?

Thank you.

Unable to use pretrained models

Hi, I've been unsuccessfully trying to reproduce your results. Neither train nor test seem to be working on my machine, all the results are almost zero. Here is what I got when I tried testing:

$ python eval.py --config_file config/gtav2cityscapes_t4/run_task/sl_3.yaml --resume_from pretrained_models/gta5_t4_best.pth 
Resume from: pretrained_models/gta5_t4_best.pth
Eval Size: [[1024, 512], [1280, 640], [1536, 768], [1800, 900], [2048, 1024]]
Use Flip: False
100%|█████████████████████████████████████████| 250/250 [11:54<00:00,  2.86s/it]
val_miou: 0.0049, 0: 0.0000, 1: 0.0001, 2: 0.0117, 3: 0.0266, 4: 0.0358, 5: 0.0062, 6: 0.0000, 7: 0.0000, 8: 0.0001, 9: 0.0054, 10: 0.0000, 11: 0.0020, 12: 0.0008, 13: 0.0008, 14: 0.0000, 15: 0.0003, 16: 0.0000, 17: 0.0029, 18: 0.0000

I will appreciate your suggestions!

Running Error

hi， Thanks for sharing teh code!
When running the code, I have the bug: "No module named apex"
Could you please provide the corresponding file "apex.parallel" in the trainer file??
Thans for your reply!

cudaGetLastError() == cudaSuccess INTERNAL ASSERT FAILED

When I use the script to train the model,the following error occurs

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/lll/anaconda3/envs/ICT_py37_torch14/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
    fn(i, *args)
  File "/home/lll/pycharm_project/IAST/code/main.py", line 71, in main_worker
    train_net(net=net, cfg=cfg, gpu=proc_idx)
  File "/home/lll/pycharm_project/IAST/code/sseg/workflow/trainer.py", line 280, in train_net
    intersection, union = intersectionAndUnionGPU(label_pred, labels, n_class)
  File "/home/lll/pycharm_project/IAST/code/sseg/datasets/metrics/miou.py", line 72, in intersectionAndUnionGPU
    area_intersection = torch.histc(intersection.float(), bins=K, min=0, max=K-1)
RuntimeError: cudaGetLastError() == cudaSuccess INTERNAL ASSERT FAILED at /tmp/pip-req-build-ufslq_a9/aten/src/ATen/native/cuda/SummaryOps.cu:253, please report a bug to PyTorch. kernelHistogram1D failed

I train it on one 1080ti with pytorch1.4 and my apex is installed by conda.The following warning is also issued at runtime:

Warning:  multi_tensor_applier fused unscale kernel is unavailable, possibly because apex was installed without --cuda_ext --cpp_ext. Using Python fallback.  Original ImportError was: ModuleNotFoundError("No module named 'amp_C'")
Warning:  apex was installed without --cpp_ext.  Falling back to Python flatten and unflatten.

Is this issue related to my APEX installation version? it seems that conda's apex is not complete, which is not include cuda_ext and cpp_ext.

training on novel dataset

Hey, great paper! any advice on how to train IAST on a new dataset?

Code

Hi, when the code will be released?

The warmup_at stage can't reproduce

hi, I meet some problem in warmup_at stage (the second stage). The miou of the model didn't increase compare to the source only model. The value of miou decreased and increased once and once and the best model is 32.63, no better than source only result. I can't find what is wrong. In addition, I find the bn layers are not frozen correctly.