KeyError: 'ViT is not in the models registry'

[ajktym94]

KeyError: 'ViT is not in the models registry'

[gsrujana]

I was able to run the base model, python demo/top_down_video_demo_with_mmdet.py demo/mmdetection_cfg/yolov3_d53_320_273e_coco.py https://download.openmmlab.com/mmdetection/v2.0/yolo/yolov3_d53_320_273e_coco/yolov3_d53_320_273e_coco-421362b6.pth configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_coco_256x192.py checkpoint/vitpose-b.pth --video-path /workplace/efsdrive/users/sgattupa/pose/datasets/video.mp4 --out-video-root vis_results_vit_b. I did not try vit-huge model. My pytorch version is 1.9.0+cu102. Installed mmdet and mmcv-full same as you did. Only difference is peerhaps I installed Vitpose after I installed mmpose and mmdet.

[ajktym94]

However, I tried this and this is working properly. But not with ViT.

python demo/top_down_video_demo_with_mmdet.py \
    demo/mmdetection_cfg/yolov3_d53_320_273e_coco.py \
    https://download.openmmlab.com/mmdetection/v2.0/yolo/yolov3_d53_320_273e_coco/yolov3_d53_320_273e_coco-421362b6.pth \
    configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
    https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
    --video-path ../UCF_Videos/Fighting/Fighting018_x264.mp4 \
    --out-video-root ../output/test1

[ajktym94]

Thanks @gsrujana !!

I managed to make it work with the following steps.

conda create -n open-mmlab python=3.8.5 -y
conda activate open-mmlab
conda install pytorch=1.10.0 torchvision cudatoolkit=10.2 -c pytorch

pip install -U openmim
mim install mmcv-full==1.5.0
pip install mmdet

git clone https://github.com/ViTAE-Transformer/ViTPose.git
cd ViTPose
pip install -v -e .

pip install timm==0.4.9 einops

Some tips for others:

I checked this page to find the compatible CUDA and PyTorch versions:
https://mmcv.readthedocs.io/en/latest/get_started/installation.html

Also, this page to find the compatible mmcv version to be installed using MIM (change the CUDA version and Torch version in the URL accordingly):
https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html

Looks like the order of installing : mmcv, mmdet and then ViTPose needs to be maintained.

[NeelayS]

Thanks @gsrujana !!

I managed to make it work with the following steps.

conda create -n open-mmlab python=3.8.5 -y
conda activate open-mmlab
conda install pytorch=1.10.0 torchvision cudatoolkit=10.2 -c pytorch

pip install -U openmim
mim install mmcv-full==1.5.0
pip install mmdet

git clone https://github.com/ViTAE-Transformer/ViTPose.git
cd ViTPose
pip install -v -e .

pip install timm==0.4.9 einops

Some tips for others:

I checked this page to find the compatible CUDA and PyTorch versions: https://mmcv.readthedocs.io/en/latest/get_started/installation.html

Also, this page to find the compatible mmcv version to be installed using MIM (change the CUDA version and Torch version in the URL accordingly): https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html

Looks like the order of installing : mmcv, mmdet and then ViTPose needs to be maintained.

Hey, when I use the script as is, I get the following error:

Traceback (most recent call last):
  File "top_down_video_demo_with_mmdet.py", line 165, in <module>
    main()
  File "top_down_video_demo_with_mmdet.py", line 125, in main
    pose_results, returned_outputs = inference_top_down_pose_model(
  File "/home/nshah/work/packages/vitpose/mmpose/apis/inference.py", line 415, in inference_top_down_pose_model
    poses, heatmap = _inference_single_pose_model(
  File "/home/nshah/work/packages/vitpose/mmpose/apis/inference.py", line 307, in _inference_single_pose_model
    return result['preds'], result['output_heatmap']
KeyError: 'preds'

The output of the model only contains a heatmap and not the actual keypoints. Did you modify the script to get the keypoints?