mediapipe_tasks_vision_image_segmenter_imagesegmentergraph__mediapipe_tasks_components_processors_imagepreprocessinggraph__ImageToTensorCalculator

[tarakang]

mediapipe_tasks_vision_image_segmenter_imagesegmentergraph__mediapipe_tasks_components_processors_imagepreprocessinggraph__ImageToTensorCalculator

[tarakang]

image_format = mp.ImageFormat.SRGB if numpy_image.shape[-1] == 4: image_format = mp.ImageFormat.SRGBA elif numpy_image.shape[-1] == 3: image_format = mp.ImageFormat.SRGB If I change this to mp.ImageFormat.SRGB, I get the following error: Traceback (most recent call last):
File "/Users/kangjian/media/mult_test.py", line 115, in
process(inputs)
File "/Users/kangjian/media/mult_test.py", line 61, in process
media_pipe_image = get_mediapipe_image(numpy_image=image)
File "/Users/kangjian/media/mult_test.py", line 53, in get_mediapipe_image
return mp.Image(image_format=image_format, data=numpy_image)
RuntimeError: float image data should be either VEC32F1, VEC32F2, or VEC32F4 MediaPipe image formats.

[kuaashish]

Hi @kinarr,

I can replicate the problem in Colab Gist, as indicated by @tarakang. I encounter the same error message: "failed: Unsupported format: 9". For now, this issue does not appear to be specific to macOS. Could you please have look into this issue? From our standpoint, it appears to be a legitimate bug.

Thank you!!

[kinarr]

@kuaashish I believe it's caused by the limited Image formats supported by MediaPipe but there should be a way to normalize the inputs so that it can passed to the model. Lemme take a look.

[kinarr]

Oh and there's no need to preprocess the inputs like this:

inputs = inputs.astype('float32')
inputs.shape = (1,) + inputs.shape
inputs /= 255

Please see the following for the correct usage of the API:

mediapipe/mediapipe/tasks/cc/vision/image_segmenter/image_segmenter_test.cc

Line 99 in 7dcf9ae

Image GetSRGBImage(const std::string& image_path) {

mediapipe/mediapipe/tasks/cc/vision/image_segmenter/image_segmenter_test.cc

Line 381 in 7dcf9ae

GetSRGBImage(JoinPath("./", kTestDataDirectory, "mozart_square.jpg"));

These are C++ API calls but it should be similar for Python.

[kinarr]

@tarakang @kuaashish Here's the working notebook for your reference: https://colab.research.google.com/drive/1B0mPPfcWCyr07CBwXgraBIkdqri5BGff?usp=sharing

[kinarr]

The model card for the MediaPipe Selfie Segmentation model provides essential information about the expected input format for the model. So PTAL here for the respective model cards for each segmentation model: https://developers.google.com/mediapipe/solutions/vision/image_segmenter#multiclass-model

As for the input features this should be helpful: https://developers.google.com/mediapipe/solutions/vision/image_segmenter#features

[kinarr]

@tarakang I've added your image utility (get_mediapipe_image) in the notebook so you can uncomment it and try it out too and it should work fine.

[kuaashish]

Hi @kinarr,

Thank you, @tarakang. It appears there may be an issue with the limited image format support in Mediapipe. Could you kindly test the provided working notebook and inform us if the issue persists?

Thank you!!

[github-actions]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.