Great Work!!!! Few queries regarding the evaluation code

[VIROBO-15]

Great Work!!!! Few queries regarding the evaluation code

[Xiaoming-Zhao]

From the error message, it seems like you do not have checkpoints at /proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372. Put your checkpoints there with structure:

+-- ckpts
|  +-- gmpi_pretrained
|  |  +-- 20220727_102024290372
|  |  |  +-- config.pth
|  |  |  +-- discriminator.pth
|  |  |  +-- generator.pth
|  |  |  +-- ema.pth
|  |  |  +-- ema2.pth

[VIROBO-15]

Thanking You @Xiaoming-Zhao for replying... I have fixed the issue but still there are some issue which I am not able to figure out

I have trained the model on FFHQ256.
Script :
bash {GMPI_ROOT}/gmpi/eval/eval.sh {GMPI_ROOT} FFHQ256 20220727_102024290372 ${Deep3DFaceRecon_PATH} debug

I have loaded the correct ema.pth file still there is some issue in loading the checkpoints..

Can you please a look at this..

Load weights from /proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/generator.pth

Load weights from /proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/ema.pth

Traceback (most recent call last):
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/prepare_fake_data.py", line 280, in
main(opt)
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/prepare_fake_data.py", line 172, in main
generator = setup_model(opt, config, metadata, mpi_xyz_input, mpi_xyz_only_z, vis_mesh=False, device=device)
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/common.py", line 144, in setup_model
ema.load_state_dict(ema_state_dict)
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_ema/ema.py", line 299, in load_state_dict
"Tried to load_state_dict() with the wrong number of "
ValueError: Tried to load_state_dict() with the wrong number of parameters in the saved state.

Compute FID/KID from
real: /proj/cvl/users/x_fahkh/mn/ml-gmpi/runtime_dataset/real_data/FFHQ256/FFHQ_real_res_256_n_50000
fake: /proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_50000/psi_1.0/fid_kid/rgb

Creating feature extractor "inception-v3-compat" with features ['2048']
Extracting features from input1
Traceback (most recent call last):
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/compute_fid_kid.py", line 29, in
verbose=True,
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/metrics.py", line 239, in calculate_metrics
featuresdict_1 = extract_featuresdict_from_input_id_cached(1, feat_extractor, **kwargs)
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/utils.py", line 372, in extract_featuresdict_from_input_id_cached
featuresdict = fn_recompute()
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/utils.py", line 360, in fn_recompute
return extract_featuresdict_from_input_id(input_id, feat_extractor, **kwargs)
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/utils.py", line 342, in extract_featuresdict_from_input_id
input = prepare_input_from_id(input_id, **kwargs)
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/utils.py", line 275, in prepare_input_from_id
return prepare_input_from_descriptor(input_desc, **kwargs)
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/utils.py", line 255, in prepare_input_from_descriptor
f'Input descriptor "input" field can be either an instance of Dataset, GenerativeModelBase class, or a string, '
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/torch_fidelity/helpers.py", line 9, in vassert
raise ValueError(message)
ValueError: Input descriptor "input" field can be either an instance of Dataset, GenerativeModelBase class, or a string, such as a path to a name of a registered dataset (cifar10-train, cifar10-val, stl10-train, stl10-test, stl10-unlabeled), a directory with file samples, or a path to an ONNX

When I am using the pertained weight the model is performing good and it calculating the FID and KID but when I train the model and use the weights the this error appears.

[Xiaoming-Zhao]

@VIROBO-15 Thanks a lot for reporting this issue. I just submitted a commit to fix this. You do not need to re-train the model since the issue only arises in the evaluation code. Hope that helps.