Questions about copy_state_dict(state_dict[key], model) in test_reid.py

[YuanXinCherry]

Questions about copy_state_dict(state_dict[key], model) in test_reid.py

[yxgeee]

The state_dict is loaded from args.resume (https://github.com/open-mmlab/OpenUnReID/blob/master/tools/test_reid.py#L89).
If you want to test any trained model, you need to define the path of the model to args.resume. (Refer to https://github.com/open-mmlab/OpenUnReID/blob/master/docs/GETTING_STARTED.md#test)

[YuanXinCherry]

Yes, I have downloaded the trained model of UDA_TP, MMT, and SpCL, and test the three model by defining the path of the model to args.resume. (Refer to https://github.com/open-mmlab/OpenUnReID/blob/master/docs/GETTING_STARTED.md#test)

My shell is:
python ./tools/test_reid.py ./logs/UDA_TP/market1501/model_best.pth --config ./logs/UDA_TP/market1501/config.yaml --launcher "none"

python ./tools/test_reid.py ./logs/UDA_TP/dukemtmcreid/model_best.pth --config ./logs/UDA_TP/dukemtmcreid/config.yaml --launcher "none"

python ./tools/test_reid.py ./logs/MMT/market1501/model_best.pth --config ./logs/MMT/market1501/config.yaml --launcher "none"

python ./tools/test_reid.py ./logs/MMT/dukemtmcreid/model_best.pth --config ./logs/MMT/dukemtmcreid/config.yaml --launcher "none"

python ./tools/test_reid.py ./logs/SpCL/market1501/model_best.pth --config ./logs/SpCL/market1501/config.yaml --launcher "none"

python ./tools/test_reid.py ./logs/SpCL/dukemtmcreid/model_best.pth --config ./logs/SpCL/dukemtmcreid/config.yaml --launcher "none"

But the mAP of UDA_TP, MMT, and SpCL all is same as Direct infer on market or duke

[YuanXinCherry]

UDA_TP, MMT, and SpCL are same
result on market1501:
Mean AP: 2.2%
CMC Scores:
top-1 6.7%
top-5 14.9%
top-10 20.1%
Testing time: 0:00:26.302841

reslut on duke:
Mean AP: 2.3%
CMC Scores:
top-1 7.5%
top-5 14.7%
top-10 18.1%
Testing time: 0:00:45.282920

[yxgeee]

It seems that you did not successfully load the trained weights. Did you use 1 GPU for testing? Try to use

GPUS=1 bash dist_test.sh ${RESUME} ${CONFIG} [optional arguments]

[yxgeee]

The problem is that the keys in state_dict all start with module., while your model is not wrapped with DistributedDataParallel or DataParallel if you use python test_reid.py --launcher "none". So please try to use GPUS=1 bash dist_test.sh.

[YuanXinCherry]

Yes，I use 1 GPU for testing. And I have tried to use
GPUS=1 bash ./tools/dist_test.sh ./logs/UDA_TP/market1501/model_best.pth ./logs/UDA_TP/market1501/config.yaml
But there are errors:
`+ PYTHON=python

• RESUME=./logs/UDA_TP/market1501/model_best.pth
• CONFIG=./logs/UDA_TP/market1501/config.yaml
• PY_ARGS=
• GPUS=1
• true
• PORT=46091
  ++ nc -z 127.0.0.1 46091
  ++ echo 127
• status=127
• '[' 127 '!=' 0 ']'
• break
• python -m torch.distributed.launch --nproc_per_node=1 --master_port=46091 --use_env test_reid.py ./logs/UDA_TP/market1501/model_best.pth --config=./logs/UDA_TP/market1501/config.yaml --launcher=pytorch --tcp-port=46091 --set
  /home/aistudio/external-libraries/open-mmlab/bin/python: can't open file 'test_reid.py': [Errno 2] No such file or directory
  Traceback (most recent call last):
  File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/runpy.py", line 193, in _run_module_as_main
  "main", mod_spec)
  File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/runpy.py", line 85, in _run_code
  exec(code, run_globals)
  File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in
  main()
  File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
  cmd=cmd)
  subprocess.CalledProcessError: Command '['/home/aistudio/external-libraries/open-mmlab/bin/python', '-u', 'test_reid.py', './logs/UDA_TP/market1501/model_best.pth', '--config=./logs/UDA_TP/market1501/config.yaml', '--launcher=pytorch', '--tcp-port=46091', '--set']' returned non-zero exit status 2.`

[yxgeee]

Please cd tools/ first, and then bash.

[YuanXinCherry]

cd tools/, then bash, there are errors:
Traceback (most recent call last):
File "test_reid.py", line 116, in
main()
File "test_reid.py", line 92, in main
test_loaders, queries, galleries = build_test_dataloader(cfg)
File "/home/aistudio/work/OpenUnReID/openunreid/data/builder.py", line 250, in build_test_dataloader
dn, data_root, "query", del_labels=False, transform=test_transformer
File "/home/aistudio/work/OpenUnReID/openunreid/data/datasets/init.py", line 47, in build_dataset
return __factory[name](root, mode, *args, **kwargs)
File "/home/aistudio/work/OpenUnReID/openunreid/data/datasets/market1501.py", line 77, in init
self.check_before_run(required_files)
File "/home/aistudio/work/OpenUnReID/openunreid/data/utils/base_dataset.py", line 128, in check_before_run
raise RuntimeError('"{}" is not found'.format(fpath))
RuntimeError: "/home/aistudio/work/OpenUnReID/OpenUnReID/datasets/market1501/query" is not found
Traceback (most recent call last):
File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in
main()
File "/home/aistudio/external-libraries/open-mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/aistudio/external-libraries/open-mmlab/bin/python', '-u', 'test_reid.py', '../logs/UDA_TP/market1501/model_best.pth', '--config=../logs/UDA_TP/market1501/config.yaml', '--launcher=pytorch', '--tcp-port=13272', '--set']' returned non-zero exit status 1.

[yxgeee]

Where did you put your datasets? Did you put it under OpenUnReID/datasets/market1501/Market-1501-v15.09.15?

[YuanXinCherry]

My dataset path is:
work/OpenUnReID/datasets/market1501/Market-1501-v15.09.15

[yxgeee]

It seems the problem is on DATA_ROOT in config file. Try to add the absolute path, e.g. GPUS=1 bash dist_test.sh ../logs/UDA_TP/market1501/model_best.pth ../logs/UDA_TP/market1501/config.yaml DATA_ROOT /home/aistudio/work/OpenUnReID/datasets.

[YuanXinCherry]

Thank you very much. It works, the result is right.
Mean AP: 34.7%
CMC Scores:
top-1 58.6%
top-5 74.0%
top-10 78.9%
Testing time: 0:00:25.071068

[yxgeee]

Glad to hear about it! Thanks for reminding me that my document is not comprehensive enough, I will expand it.