ascend-research/headposeestimation-whenet

headposeestimation-whenet's Issues

How to evaluate on BIWI dataset?

Hi, I want to test the BIWI dataset with your method, which model can i use for the results '3.8 MAE' of your paper?

how can i get the 3D position of head (tx,ty,tz)

Hi , hope for your answer.

Can use weight for commercial ? please answer me

BIWI dataset

can you share original BIWI dataset again？

Angle wrong from mtc data

Some of the angle (pitch, yaw, roll) parsing from mtc data is obviously wrong. Do you ever met this phenomenon?

Error in runing demo_video.py

Hi I am runing the demo_video.py, and get the following error:

Traceback (most recent call last):
  File "demo_video.py", line 81, in <module>
    main(args)
  File "demo_video.py", line 54, in main
    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(3.4.8) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Can you help me ? Thanks a lot!

How to convert to the onnx model

I tried to convert keras2onnx and tf2onnx into the onnx model, but failed。

Config folder is missing in MTC dataset

Hi,
Thank you for the great work and amazing codebase. I have a question regarding dataset preparation.
1. Line 215 in prepare_images.py, the config folder is missing in the MTC dataset. Although the downloaded dataset has the same size(270GB) as mentioned in documentation. There is no mention of this config folder in official website also. Any help is appreciated.

loss function

Hey,

Can you share the loss function implementation which you use for your paper? I want to reimplement your paper.

reading from ip camera

Hi I could not to read from ip camera? could you guide me?

about visualize(plotting)

draw_axis in utils.py

# X-Axis pointing to right. drawn in red
x1 = size * (cos(yaw) * cos(roll)) + tdx
y1 = size * (cos(pitch) * sin(roll) + cos(roll) * sin(pitch) * sin(yaw)) + tdy

# Y-Axis | drawn in green
x2 = size * (-cos(yaw) * sin(roll)) + tdx
y2 = size * (cos(pitch) * cos(roll) - sin(pitch) * sin(yaw) * sin(roll)) + tdy

# Z-Axis (out of the screen) drawn in blue
x3 = size * (sin(yaw)) + tdx
y3 = size * (-cos(yaw) * sin(pitch)) + tdy

I don't understand how did you get these formula.

could you give me some references ?

I met a problem when loading the weight file.

I tried to use the demo_video.py file, but ''ValueError: You are trying to load a weight file containing 133 layers into a model with 134 layers.'' error occurred and cannot proceed more. Do you have any idea why the weight file doesn't match the layers of the model?

trainning

Can you provide the training code for this model? Thanks you

About cls function.

Hello, a sigmoid activation with binary cross entropy is selected for the classification loss function mentioned in your paper. If this is done, the classification part will become multilable classification, but multilable classification deals with the problem of non mutually exclusive multi classification. Angle classification belongs to the problem of mutually exclusive multi classification. I don't quite understand this concept. Can I have an answer? thank you!

Very Painful to set environmet and install dependence.

Very Painful to set environment and install dependence.

provide head_detect.h5

Hi, Can you provide the head_detect.h5 pretrained model? Thanks you

Is that possible to share the training dataset : CMU Panoptic Dataset ?

Please revise requirements.txt: h5py==2.10.0

Please add the line in requirement.txt because the tensorflow-gpu 1.12.0 and keras 2.1.6 cannot use the latest h5py.

h5py==2.10.0

wrap loss

在我重新实现你的论文的时候，我找到了你的包裹损失函数的缺点，并想了一个改进方法，但是不确定你是否解决了，能给个微信或邮箱，或者其他联系方式以便于我们细聊吗?
When I reimplemented your thesis, I found the shortcomings of your wrap loss function and thought of a way to improve it, but I am not sure if you have solved it, can you give us a WeChat or email, or other contact information so that we can talk in detail ?

try to reproduce the small aflw error

so i was trying to reproduce whenet-v using the same network structure, preprocessor, etc. i could only get around 30 yaw loss and around 10 pitch and roll loss for AFLW2000, even after filtering out angles greater than 99 degrees. i saw in your paper that whenet-v was trained on 300w and got an MAE on AFLW2000 below 5.
i used hopenet's preprocessor for 300W and a batch size of 16, 1e-5 learning rate, same network structure, 25 episodes to train on 300W, etc. what should i check / change to reproduce the small AFLW error?

also in your opinion can be model be further compressed to say like a few hundred kilobytes (assuming more training data, etc.) with a small loss of accuracy in practice? thanks.

Error

This is my error. Can you help me ?
" Traceback (most recent call last):
File "demo.py", line 21, in
model = WHENet('WHENet.h5')
File "/home/hoangnn/pose/HeadPoseEstimation-WHENet/whenet.py", line 13, in init
out = keras.layers.GlobalAveragePooling2D()(out)
File "/home/hoangnn/.local/lib/python3.7/site-packages/keras/engine/topology.py", line 606, in call
previous_mask = _collect_previous_mask(inputs)
File "/home/hoangnn/.local/lib/python3.7/site-packages/keras/engine/topology.py", line 2845, in _collect_previous_mask
mask = node.output_masks[tensor_index]
AttributeError: 'Node' object has no attribute 'output_masks'"

large MSE of the pretrained model on AFLW dataset?

I test you provided pretrained model on aflw data from FSA.
I get result: [yaw 5.98914401 pitch 7.53006404 roll 5.87366357]

Inaccurate results for profile faces

Hi,

I'm currently testing the head pose estimation for profile-view faces, however, I am attaining unusual results.
For test purposes, I'm using the 3 following profile view images:

For these 3 images, I get the following results:
Yaw: [70.59546] Pitch: [34.594513] Roll: [33.04599]
Yaw: [81.640686] Pitch: [-8.637505] Roll: [-13.95977]
Yaw: [71.53331] Pitch: [10.675278] Roll: [13.468033]

I am mainly focused on the yaw angles - It is clearly evident that these images contain strictly profile view images, however, neither of the detected yaw angles are close enough to 90 degrees.

If anyone has any ideas as to what might be causing this issue, it would be highly appreciated.

Thanks!

change to yolov5

hi can I ask how do you make yolov3 fit your model? I want to change it to Yolov5

Swish activation layer error

Hi, i have converted this model to uff and onnx, but in both cases I am not able to serialize to a trt engine due to the swish layer.

Error message:
ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin swish_f32 version 1
ERROR: builtin_op_importers.cpp:3773 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
ERROR: Network must have at least one output
ERROR: Network validation failed.

kindly guide solutions or alternatives if possible, thanks
p.s: the model is trained really well! i tested on faces with masks as well, very promising :)

Cant get it up and running, any help is appreciated

I cant get this project up and running, would it be possible to get some more instructions ?

What python version are you using?

Any other dependencies ?

What CPU and OS are you running ?

using CUDA ?

Tensorflow version

Hello. I cannot install tensorflow-gpu 1.12.0 using pip.
What other versions are available?
Thanks you

Error: This application failed to start because no Qt platform plugin could be initialized.

Hi I am using a Docker container to run this model, but it is throwing me Aborted (core dumped) error.

qt.qpa.xcb: could not connect to display :1 qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/usr/local/lib/python3.5/dist-packages/cv2/qt/plugins" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem

this is how I am using the command,
python3 demo_video.py --video off.mp4 --display 0 --output output.mp4

Seems like problem is with --display argument.

It take so much time on this step.

about fps

in paper, WHENet has achieved 60fps. what a fast inference!
I'm wondering what environment did you achieve that fps?