mediapipe hands not working over grayscale images even though I am stacking it to make it 3 channel.

[Suvi-dha]

mediapipe hands not working over grayscale images even though I am stacking it to make it 3 channel.

[kuaashish]

Hi @Suvi-dha,

Based on the code snippet you've shared, it seems you're utilizing an older version of MediaPipe. We've recently introduced improved MediaPipe Tasks APIs that offer greater stability and functionality compared to the older version.

We highly recommend transitioning to these updated APIs for Hand landmarks, as we've ceased support for legacy solutions. You can access the documentation for the Hand Landmarker here.

However, it's worth noting that the newer version does support Grayscale images. You can refer to the example gist demonstrating detection on a grayscale image.

Thank you!!

[Suvi-dha]

Hi, Thanks for the input. As asked, I tried and tested the example notebook on my samples and I am still getting empty lists like this. HandLandmarkerResult(handedness=[], hand_landmarks=[], hand_world_landmarks=[]). Can you please look at the examples I attached in the drive link I posted above. May be mediapipe is not good at those kind of image.

[kuaashish]

Hi @kinarr,

Could you please have look into this issue?

Thank you!

[kinarr]

I'll take a look.

…

On Fri, Mar 15, 2024, 11:26 AM kuaashish ***@***.***> wrote: Hi @kinarr <https://github.com/kinarr>, Could you please have look into this issue? Thank you! — Reply to this email directly, view it on GitHub <#5226 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKGYHL3QH35FD4KGOLVTXMLYYKEPBAVCNFSM6AAAAABEVOC6KCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJYHE4TEMZWHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kinarr]

Hi, @Suvi-dha
Here's the notebook for exploring the results for those samples. It appears there's a challenge with the model generating hand landmarks in images that are too zoomed in, making it difficult to identify details. You might consider using images where the hand is more visible and less zoomed in.

https://colab.research.google.com/drive/1Ejd67MUnHsdIsgH56YgX1uSd_9incPWU

[Suvi-dha]

Thank you for your response, appreciate your help. Looks like mediapipe needs more generalization.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[kinarr]

Thank you for your response, appreciate your help. Looks like mediapipe needs more generalization.

@Suvi-dha If the team releases a MediaPipe Model Maker guide for Hand Landmarker, you can fine-tune the model with your own images, similar to the Gesture Recognizer's Model Maker guide.

https://developers.google.com/mediapipe/solutions/customization/gesture_recognizer

At the moment there's no Hand Landmarker Customization guide.

[kinarr]

You can drop your images into MediaPipe Studio and adjust the relevant parameters to obtain reliable results, which helps determine if the model is suitable for your specific conditions. Additionally, you can insert a custom fine-tuned Hand Landmarker model, but currently, MediaPipe Model Maker does not support the HandLandmarker API.

https://mediapipe-studio.webapps.google.com/studio/demo/hand_landmarker