jhb86253817 / pipnet Goto Github PK
View Code? Open in Web Editor NEWEfficient facial landmark detector
License: MIT License
Efficient facial landmark detector
License: MIT License
What is the num_nb
?
Some parts of the face may be occluded. The key points should be predicted. Can you get the credibility value?
error D8021
error: command 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\HostX86\x64\cl.exe' failed with exit code 2
Is FaceBoxesV2 compatible with windows or am I doing something wrong?.
Thanks in advance.
Hello, I was working on the LaPa dataset part of your landmark detection model. You shared the pretrained model of LaPa dataset but preprocess part is empty which is important for training. I know that preprocess is cropping the faces from dataset images and transforming them into 256x256 size. However, LaPa dataset does not contain face bounding coordinates unlike other datasets. How did you train your model without cropping images? Or did you manually select the face box coordinates?
Can the project be deployed on WINDOWS?
Hello, when I use train_gssl.py, "300W nme is inf", is this normal?
300W nme: inf
COFW nme: 0.05221280819123009
WFLW nme: 0.08741295886838747
Hello, author thanks for sharing your code.
i want to training my custom dataset, but when i run train.py, i have an Error named 'ModuleNotFoundError: No module named 'PIPNet' '. so how can i get PIPNet module?
Is the current code ready to train now?
Hi, this is my first time asking questions on GitHub, please point out if anything is unclear
After changing the parameters in Config(), I cannot change them again (num_nb). My config is shown below:
class Config(): def __init__(self): self.det_head = 'pip' self.net_stride = 32 self.batch_size = 16 self.init_lr = 0.0001 self.num_epochs = 60 self.decay_steps = [30, 50] self.input_size = 256 self.backbone = 'resnet18' self.pretrained = True self.criterion_cls = 'l2' self.criterion_reg = 'l1' self.cls_loss_weight = 10 self.reg_loss_weight = 1 self.num_lms = 4 self.save_interval = self.num_epochs self.num_nb = 0 self.use_gpu = True self.gpu_id = 0
But when running run_train.sh, it showed the last parameter.
experiment_name: pip_32_16_60_r18_l2_l1_10_1_nb10 data_name: CFP det_head: pip net_stride: 32 batch_size: 16 init_lr: 0.0001 num_epochs: 60 decay_steps: [30, 50] input_size: 256 backbone: resnet18 pretrained: True criterion_cls: l2 criterion_reg: l1 cls_loss_weight: 10 reg_loss_weight: 1 num_lms: 4 save_interval: 60 _num_nb: 10_ use_gpu: True gpu_id: 2
I was confused for a long time,so I came to ask you.
Hi!
Thank you very much for this and making it public!
I have tried running the training for ResNet101 for 300W dataset (I am using Python on WSL with Ubuntu distro) and I have been getting this error.
"Traceback (most recent call last):
File "lib/train.py", line 32, in
my_config = importlib.import_module(config_path, package = 'PIPNet')
File "/home/apecundo725/anaconda3/envs/PIPNet/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 970, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'PIPNet.experiments.data_300W.pip_32_16_60_r101_l2_l1_10_1_nb10.'; 'PIPNet.experiments.data_300W.pip_32_16_60_r101_l2_l1_10_1_nb10' is not a package"
I have kept the folder structure as is from the github but I keep getting this error. I am wondering what fixes can be applied (I have tried adding init.py files but it does not help)
I have been able to run the preprocessing.py script but am stuck at the training part of the process.
Thank you very much for the support.
Hi
Thank you for sharing your codes.
I have downloaded one of the dataset (COFW) and tested using available pre-trained models (resnet18). The NME score that I got is 0.08758. If I multiply the score with 100%, it results in 8.758%.
Q: How to produce the same NME score as your paper? In the COFW, it is reported as 3.31%. Thank you.
Hello, the url(https://drive.google.com/drive/folders/1WruSmUzb2mqZnSiqWO8wRWZcttaxmNnZ) of snapshot is no longer accessible.
First of all, thanks for sharing the excellent models. I've tested the resnet101 based models, it works perfect. And I noticed that you guys also provide the mobilenet-v3-large model (mobilenetv3-large-1cd25616.pth) in ~/PIPNet/lib
.
I'd like to test the effect of this model, but can't find the related cfg file, seems that only Resnet related cfg files have been provided.
Could you provide the 'cfg' file and also the 'data_name' which may needed in calculating the get_meanface
function.
Hi,I only use the eyes and mouth parts of the key point to train PIPNET, but there is a strange problem.
I used the same model to predict key points in two videos, one of which predicted eye and mouth key points normally, while the other predicted key points at the edge of the face
File "/home/inspur/Projects/PIPNet/lib/functions_gssl.py", line 85, in compute_loss_pip │ outputs_local_x_select = torch.gather(outputs_local_x, 1, labels_max_ids) │ RuntimeError: index 18 is out of bounds for dimension 1 with size 16
when I use train_gssl.py, the problem comes. Below is my config
`# with curriculum
class Config():
def init(self):
self.det_head = 'pip'
self.net_stride = 32
self.batch_size = 16
self.init_lr = 0.0001
self.num_epochs = 60
self.decay_steps = [30, 50]
self.input_size = 256
self.backbone = 'resnet18'
self.pretrained = True
self.criterion_cls = 'l2'
self.criterion_reg = 'l1'
self.cls_loss_weight = 10
self.reg_loss_weight = 1
self.num_lms = 68
self.save_interval = self.num_epochs
self.num_nb = 10
self.use_gpu = False
self.gpu_id = 1
self.curriculum = True`
I'm trying to use the PIPNet framework to train a custom model unrelated to human faces with a non-standard number of keypoints (4). I'm confused about this line in lib/train.py:
labels = get_label(cfg.data_name, 'train.txt')
what is labels, what format is it and what is train.txt?
i confuse about : mu_x = int(floor(target[i][0] * map_width)),why target[i][0] * map_width not target[i][0]
I trained on my dataset and use the input size 64X80, and found the NME is larger than the regression method before used, and when I modified the input size to 256X256, NME is lower than before. @jhb86253817
Hello, thank you for your work. Is it possible to train the model with additional custom data, where not all the points have Ground Truth positions? For example, I want to train for 68 landmarks, but several images from my custom data are only annotated with 40 points and the remaining 28 points are not annotated.
hi~ I have found that you only use 19 of 21 landmarks in AFLW, and point out that it's follow CPW. But, I can't found any information about AFLW setting in CPM's paper. Could you please release the detail indices of 19 landmarks you have used in you paper?
I see the code in train.py
:
points_flip = [6, 5, 4, 3, 2, 1, 12, 11, 10, 9, 8, 7, 15, 14, 13, 18, 17, 16, 19] # no 0 and 20
points_flip = (np.array(points_flip)-1).tolist() # then from 0~18, total 19 points
so, it means that you have exclude 0 and 20 in AFLW ?
hello, the url((https://drive.google.com/drive/folders/17OwDgJUfuc5_ymQ3QruD8pUnh5zHreP2?usp=sharing) cann't download, thanks!
Hi,I'm sorry, I still don't understand,
the max_cls returned by function forward_pip is the confidence scores of predicted landmarks.
why is the confidence of the occluded points higher than that of the unoccluded points?
For example, if the face is partially occluded, I want to know which key points are occluded,Or the occluded key points are not displayed.
Can it be achieved based on the confidence of key points?
Hi, thanks for great works.
I have some question, if only eye, eyebrow, nose part for training, accuracy is increasing? my concern is large pose and occlusion image make some problem like lower performance even if using resnet101. Do u have some idea? If I using multiple dataset, will it be good?
thanks
hey,thanks to your contritubution for keypoints detection! I wanna know how to train my own datasets? I just wanna detect 2 keypoints,instead of 68 keypoints.
For inference, we still have some image that have landmark prediction very bad which has wrong face shape (compare with regression method)? Could you suggest any way to filter the result ? Note that, I am using video as input and landmark score that we used may not good
In order to shorten the detection time of the face detect, I predict the face box of the current frame by using the landmark of the previous frame of the image. In some cases, the landmarks predicted by PIPNet predicted would appear outside my face box.
In the image below, the area in the red box is the image I entered into the network, and the green dot is the landmarks predicted by PIPNet.
Some of these landmarks appear outside the box, does anyone know why?
Hello, it is very nice work! I have some question about curriculum training.
I have attention that the line 239-242 in train_gssl.py,that mean every loop in task_type_list you should build a new network model in the code ? And I have no find current loop training loading last loop saved weight?
this way can promise the model learning easy to hard?
Looking forward to your reply.
Official WFLW only provide 98 landmarks annotation, could you please give me someway to download the WFLW-68 annotations, or share your code about generate that? Thank u so much!
Hi are you planning to release your newer code for BarrelNet?
Line 101 in b9eab58
Should line101 be changed to:
det_ymin -= int(det_height * (det_box_scale - 1) / 2)
same in demo.py
I download all files from CelebA, but I do not find celeba_bboxes.txt. So I want to ask you where it is. Thank you very much.
Hi, when I run meanface_indices, _, _, _ = get_meanface(os.path.join('../data', 'data_300W', 'meanface.txt'), 68) it reported indexError:
File "/work/landmark/PIPNet/lib/data_utils.py", line 170, in
meanface_indices, _, _, _ = get_meanface(os.path.join('../data', 'data_300W', 'meanface.txt'), 68)
File "/work/landmark/PIPNet/lib/functions.py", line 53, in get_meanface
meanface_indices_reversed[meanface_indices[i][j]][0].append(i)
IndexError: index 67 is out of bounds for axis 0 with size 67
Very practical work! It helps a lot!
Hello, I would like to consult the relevant details of gssl training. May I ask whether a net is initialized for each task when using gssl training in the paper, and is the latest training net predicted for each pseudo-label?
My understanding is that this line of code is to get the index of the 10 coordinate points of the NMS.
But where did this initial file(,'meanface.txt') come from?
Please point out if I understand wrong, thank you!
meanface_indices, reverse_index1, reverse_index2, max_len = get_meanface(os.path.join(rootpath,'data', cfg.data_name, 'meanface.txt'), cfg.num_nb)
could i change to 68 keypoints model, or is 68 facial keypoint data somehow related to 98key point?
Hi, it is very nice work! I get some question when general the labels for train. In the paper, only one point in the score map is set 1, others are set 0, and one point in the offset map is set offset,others are set 0. In the inference phrase, how to promise all the score maps predicted are right? if there is one score map predicted is wrong, which could lead a wrong index for offset map in testing, the final landmark coordinate couldn't be right! why not set all possible offset in the gird?
Hello, I have a batch of expanded images and corresponding ldmk annotations. I want the model to be trained both on 300W and my own data at the same time. How should I do?
HI
I am trying to test the model through sh demo.sh.
However, when I run it, I get the following error:
File "/home/main/project/PIPNet/lib/functions.py", line 185, in forward_pip
tmp_y = (torch.floor(max_ids/tmp_width)).view(-1,1).float()+outputs_y_select.view(-1,1).float()
RuntimeError: _th_floor_out not supported on CUDAType for Long
I am wondering if it is a version error.
My torch version is 1.4.
Thank you.
Hello, sorry to borther you again. I train the resnet18 with "data_300W_CELEBA/pip_32_16_60_r18_l2_l1_10_1_nb10_wcc.py" , when the eyes are open, the result is good as your model, but when eyes are closed, the eyes keypoints are bad. Are there some tricks to improve the accurate of eyes keypoints
from .nms.cpu_nms import cpu_nms, cpu_soft_nms ModuleNotFoundError: No module named 'utils.nms.cpu_nms'
I've run make.sh, but still get this error
Hello, author, thanks for your sharing.
Can this network used to test CelebA-HQ-img? I want found a network to calculate landmark loss in CelebA-HQ-img, but after I seen your code, I don't konw how to test in CelebA-HQ-img.
How can I get your pre-trained 98 key points model?
which is showed in run_demo.sh (r18)
Hi, do u have tf model or tflite model?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.