Query Regarding the evaluation code for FFHOQ512

[apple]

Query Regarding the evaluation code for FFHOQ512

[Xiaoming-Zhao]

I re-run the evaluation on my side and cannot reproduce this error.

Looking for samples non-recursivelty in "/proj/cvl/users/x_fahkh/mn/ml-gmpi/runtime_dataset/real_data/FFHQ512/FFHQ_real_res_512_n_50000" with extensions png,jpg,jpeg Found 71 samples

It seems like you only have 71 images under the path /proj/cvl/users/x_fahkh/mn/ml-gmpi/runtime_dataset/real_data/FFHQ512/FFHQ_real_res_512_n_50000. However, you are trying to evaluate 50k images.

Maybe you can delete /proj/cvl/users/x_fahkh/mn/ml-gmpi/runtime_dataset/real_data/FFHQ512/FFHQ_real_res_512_n_50000 and re-run the command to generate real images.

Note, if the real_path exists, the code will not re-generate real images since generating 50k images is costly. So please double check whether you interrupt the running such that the real-image generation is not complete:

ml-gmpi/gmpi/eval/prepare_real_data.py

Line 43 in c809703

if not os.path.exists(real_dir):

Some suggestions:

1. You may set the flag from nodebug to debug. In this way, you will only generate 2048 images for FID/KID, which will be much quicker and you can use this to check the path setup. However, keep in mind, 2048 is too small to provide a reasonable FID/KID evaluation.

2. I recommend not running the whole eval.sh at first. I have split the script in a command-by-command style. Just run one command at a time. After each command, double-check the anticipated path's content. I think in this way, you will easily pinpoint the issue if there are any.

[fangchangma]

@VIROBO-15 FYI - the readability of your issues would improve, if proper code formatting is applied.

[VIROBO-15]

Thank You for replying @fangchangma @Xiaoming-Zhao...

I am still facing some issues in evaluation
Script : # for Depth and pose

eval "$(conda shell.bash hook)"
conda activate gmpi
export MKL_NUM_THREADS=5 && export NUMEXPR_NUM_THREADS=5 &&
python ${REPO_DIR}/gmpi/eval/prepare_fake_data.py
--ckpt_path ${CKPT_DIR}/generator.pth
--seed ${SEED}
--save_dir ${GEO_DIR}
--nplanes ${N_PLANES}
--n_imgs ${N_GEO}
--task geometry
--exp_config ${CKPT_DIR}/config.pth
--save_depth 1
--dataset ${DATASET}
--truncation_psi ${TRUNCATION_PSI}
--stylegan2_sanity_check ${SANITY_CHECK}

eval "$(conda shell.bash hook)"
conda activate mtcnn_env
export MKL_NUM_THREADS=5 && export NUMEXPR_NUM_THREADS=5 &&
python ${REPO_DIR}/gmpi/eval/prepare_face_landmarks.py
--data_dir ${GEO_DIR}/geometry/rgb

eval "$(conda shell.bash hook)"
conda activate deep3d_pytorch
cd ${Deep3DFaceRecon_PATH}
python ${Deep3DFaceRecon_PATH}/estimate_pose_gmpi.py
--name=pretrained
--epoch=20
--gmpi_img_root ${GEO_DIR}/geometry/rgb
--gmpi_depth_root ${GEO_DIR}/geometry/depth
--gmpi_detect_root ${GEO_DIR}/geometry/rgb/detections

eval "$(conda shell.bash hook)"
conda activate gmpi
python ${REPO_DIR}/gmpi/eval/compute_geometry.py
--geo_dir ${GEO_DIR}/geometry
--angle_err 1

Error
sorted_f_list: 1024 ['/proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_1024/psi_1.0/geometry/rgb/000000.png', '/proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_1024/psi_1.0/geometry/rgb/000001.png', '/proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_1024/psi_1.0/geometry/rgb/000002.png', '/proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_1024/psi_1.0/geometry/rgb/000003.png', '/proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_1024/psi_1.0/geometry/rgb/000004.png']

100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 1024/1024 [00:46<00:00, 22.09it/s]
/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/eval.sh: line 158: cd: nodebug: No such file or directory
python: can't open file 'nodebug/estimate_pose_gmpi.py': [Errno 2] No such file or directory
0%| | 0/1023 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/compute_geometry.py", line 93, in
tmp_depth_err = compute_depth_err(tmp_aligned_depth_f, tmp_pred_depth_f, tmp_pred_mask_f)
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/gmpi/eval/compute_geometry.py", line 26, in compute_depth_err
depth = np.load(alinged_depth_f)
File "/home/x_fahkh/.conda/envs/gmpi/lib/python3.7/site-packages/numpy/lib/npyio.py", line 417, in load
fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: '/proj/cvl/users/x_fahkh/mn/ml-gmpi/ckpts/gmpi_pretrained/20220727_102024290372/planes_96_n_1024/psi_1.0/geometry/recon/aligned_depth/000000.npy'

[Xiaoming-Zhao]

@VIROBO-15 I recommend reading the trace log if possible:

python: can't open file 'nodebug/estimate_pose_gmpi.py': [Errno 2] No such file or directory

I guess estimate_pose_gmpi.py is not under nodebug. Instead, it should be in the folder for Deep3DFaceRecon_PATH:

ml-gmpi/gmpi/eval/eval.sh

Line 159 in c809703

python ${Deep3DFaceRecon_PATH}/estimate_pose_gmpi.py \

[VIROBO-15]

Thank you for reply......

I am Still facing some issues for calculating the depth and pose score..

Error:
Traceback (most recent call last):
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/Deep3DFaceRecon_pytorch/estimate_pose_gmpi.py", line 6, in
from options.test_options import TestOptions
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/Deep3DFaceRecon_pytorch/options/test_options.py", line 4, in
from .base_options import BaseOptions
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/Deep3DFaceRecon_pytorch/options/base_options.py", line 9, in
import models
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/Deep3DFaceRecon_pytorch/models/init.py", line 22, in
from models.base_model import BaseModel
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/Deep3DFaceRecon_pytorch/models/base_model.py", line 9, in
from . import networks
File "/proj/cvl/users/x_fahkh/mn/ml-gmpi/Deep3DFaceRecon_pytorch/models/networks.py", line 18, in
from .arcface_torch.backbones import get_model
ModuleNotFoundError: No module named 'models.arcface_torch'

[Xiaoming-Zhao]

Have you completed the setup of Deep3DFaceRecon_pytorch following the steps of its README?