Cannot estimate normal map and displacement map and cannot check the output

[apchenstu]

Cannot estimate normal map and displacement map and cannot check the output

[apchenstu]

Sorry, I tried to remove cv2 libs and forgot to test the code, thanks for you help, I already fix the bugs you pointed out, please try again :)

If you don't want to re-download the new released package, you may need to

1. re-install BFM2017 with a new version of texture_uv.mat
2. re-download facialDetails.py and proxyPredictor.py.

[ak9250]

@apchenstu I get this error
Load preTrain model done.
Load preTrain model done.
Traceback (most recent call last):
File "facialDetails.py", line 294, in
main(args)
File "facialDetails.py", line 237, in main
fit_landmarks(args.landmark_exe_path, args.img_path)
File "facialDetails.py", line 57, in fit_landmarks
os.chdir(landmark_exe_path)
FileNotFoundError: [Errno 2] No such file or directory: ./landmarks

[apchenstu]

You may used an old version, this error is due to I renamed some folders. You can try the newest one release package, Thanks.

[ak9250]

@apchenstu I used the released package attached, I think the problem is with the exe files as i am trying to setup the project in google colab and currently it is setup to work only in windows
Load preTrain model done.
Load preTrain model done.
sh: 1: FaceLandmarkImg.exe: not found
===> Landmarks detection done.

===> estimating proxy of /content/Facial_Details_Synthesis/src/test.jpg
Traceback (most recent call last):
File "/usr/lib/python3.6/shutil.py", line 550, in move
os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/content/Facial_Details_Synthesis/src/landmarkDetector/processed/SH3mfFA_.box' -> '/content/Facial_Details_Synthesis/src/result/SH3mfFA_/SH3mfFA_.box'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "facialDetails.py", line 294, in
main(args)
File "facialDetails.py", line 248, in main
move_landmark(args.landmark_exe_path,save_path,base_name[0])
File "facialDetails.py", line 51, in move_landmark
shutil.move(os.path.join(landmark_exe_path,'processed',name+'.box'),os.path.join(save_path,name+'.box'))
File "/usr/lib/python3.6/shutil.py", line 564, in move
copy_function(src, real_dst)
File "/usr/lib/python3.6/shutil.py", line 263, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.6/shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/content/Facial_Details_Synthesis/src/landmarkDetector/processed/SH3mfFA_.box'

[apchenstu]

Yeah, I didn't test the code on other platforms. Your error looks like the landmarkDetector not works well. I will write the compiling guideline in those two days, maybe you can try to re-compile the code in google colab by then. Thanks

[j20232]

Sorry, I tried to remove cv2 libs and forgot to test the code, thanks for you help, I already fix the bugs you pointed out, please try again :)

If you don't want to re-download the new released package, you may need to

1. re-install BFM2017 with a new version of texture_uv.mat
2. re-download facialDetails.py and proxyPredictor.py.

Thank you for your quick response, but there are still many bugs.

1. The output part of the displacement map is wrong.

As I pointed out before, line 280 - 282 in facialDetails.py are wrong.
This part outputs the error like belows.

===> predicting details of ...\FacialDetailsSynthesis\samples\details\019615.jpg
Traceback (most recent call last):
  File "...\PIL\PngImagePlugin.py", line 771, in _save
    rawmode, mode = _OUTMODES[mode]
KeyError: 'I;16'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "facialDetails.py", line 297, in <module>
    main(args)
  File "facialDetails.py", line 282, in main
    Image.fromarray(displacementMap.astype('uint16')).save(save_path)
  File "...\PIL\Image.py", line 1994, in save
    save_handler(self, fp, filename)
  File "...\PIL\PngImagePlugin.py", line 773, in _save
    raise IOError("cannot write mode %s as PNG" % mode)
OSError: cannot write mode I;16 as PNG

2. Pretrained weights may be wrong.

The pretrained weights of DFDN may be wrong because the estimation of the normal and distplacement map doesn't work.
The outputs contain many block noises.

3. Linking the output material file with the normal and displacement map isn't correct.

result.mtl is always the following format.

newmtl FaceTexture
map_Kd result.isomap.png

It doesn't contain the normal and displacement map.

4. the path of hmrenderer.exe is wrong

Line 291 in facialDetails.py doesn't work well because the renderer cannot find the material file.

cmd = '%s/hmrenderer.exe %s %s %s'%(args.face_render_path,save_obj_path+'.obj',save_path,args.face_render_path+'/shaders')

So, the output of the rendering shows only the mesh and doesn't show the albedo.

I'm sorry many times.

Could you check them?

[apchenstu]

No worries, I noted the bugs you pointed out, actually, these four bugs are the same problem, it is because the code could not correctly load the input image.
Maybe we use different version PIL libs(PIL 6.0.0 in my PC), and the anaconda default version is 'PIL 6.1.0' now, I will fix this. Thanks.

[apchenstu]

I just setup the environment and test the code in a new PC, it can correctly output normalMap and displacementMap, for the noises output, can you show me the results, I also got noisy result before I fixed the bugs you point out, it's because value range (i.e., opencv range 0-255 and PIL range 0-1.0 after transform RGB to HSV), please check this line here.

Please ignore material warming as our phong shading model would not use material and texture.

[ak9250]

@apchenstu thanks looking forward to compile guide to try to get it to work in google colab

[j20232]

@apchenstu Thank you very much. I could try your project.