After detection whensomeone is not infront of camera ,skeltons is still there

[creativeikep]

After detection whensomeone is not infront of camera ,skeltons is still there

[creativeIKEP]

Hello. @ramkeshshadowcast .

For multiple person skeltons move one person to another . Can we do to stay on single person ?

BlazePoseBarracuda (and Mediapipe Pose, which base of BlazePoseBarracuda) is supported only one person pose estimation.
So, BlazePoseBarracuda detect random one person when multiple person are visibled.

After detection when user move away from infront of camera ,skelton is still there . I mean creat and destroy . What can be done to not generate if user is not there .

BlazePoseBarracuda output human pose visibilty and landmarks visibilities in every frames. (Please read comment outs in README.md)
So, you can set threshold, and you can operate that do not draw the skelton when human pose or landmarks visibilities are lower than threshold.
The skelton line color is blue if human pose visibilty is lower than threshold in visualize demo in this repository.
Also, The skelton line alpha color is determinated by landmarks visibilities in visualize demo in this repository.

[ramkeshshadowcast]

Thanks @creativeIKEP . I got it but it is in Shader scripts . How i can determine isHumanExist >= _humanExistThreshold) ? float4(0, 1, 0, score) : float4(0, 0, 1, score) in PoseVisuallizer Scripts ?.The value _keypointCount which Mediapipe value taking ? .

[creativeIKEP]

@ramkeshshadowcast
PoseVisuallize.cs script operates procedural drawing with GPU.
So, values in demo shader scripts are determined by PoseVisuallize.cs script.
The threshold value (that is _humanExistThreshold in shader script) is determined by humanExistThreshold variable in PoseVisuallize.cs script, and humanExistThreshold variable is controlabled in Unity Editor Inspector window.

The value _keypointCount is set the value of vertexCount variable in BlazePoseBarracuda Package (= vertexCount variable in PoseLandmarkBarracuda sub Package), and it is 33 that is Mediapipe pose landmarks count.

[ramkeshshadowcast]

Thanks @creativeIKEP . I tried to explore given terms and write code but having issue.


In PoseVisuallize.cs i want to achieve what we are doing in Shader script [Human Exist then green skelton not exist then blue ]. Sharing screenshot of both . In my tried code getting execption[ Red Arrow at line ]. Can you please guide me what i am doing wrong and how i can achieve it ?

[creativeIKEP]

@ramkeshshadowcast
I think that var data1 = new Vector4[count]; row is problem.
How many values are in the count variable?
Length of detector.outputBuffer is 34 (33 landmark datas + 1 score data).
IndexOutOfRangeException is happened if the count variable is 33 or lower.

So, you should convert to 34 for length of the data1 variable.
Or, you should get only score with below code.

var data1 = new Vector4[1];
result.GetData(data, 0, 33, 1);
float v = data1[0].x;

refarence: https://docs.unity3d.com/ScriptReference/ComputeBuffer.GetData.html

Thanks.

[ramkeshshadowcast]

@creativeIKEP Thanks a lot . It is really very helpfull .Now it's working fine.
I have one question regaring coordinate system . Accoding to mediapipe documentation it's take hip as orgin . How these coordinates can communicate with Ui screen coordinates ? . I mean is there any way to converts these points in screen coordinates or world coordinates system of unity ? I was trying to get skelton world coordninates [skeloton which show us in scene] but Graphics.DrawProceduralNow generate it in GPU . Not getting a gameobject with transforms .

[creativeIKEP]

@ramkeshshadowcast
BlazePoseDetecter.outputBuffer is data for 2D estimation and (x, y) cordinate value are normilized to [0, 1] by the image width and height.
BlazePoseDetecter.worldLandmarkBuffer is data for real-world 3D cordinate estimation and (x, y) are landmarks with the origin at the center between hips.
2 different outputs exist.

So,
I show landmark point with converting from [0, 1] to clip position in the 2D visualize demo(2DSampleScene.unity).
I show landmark point with converting from 3D cordinate to clip position in the 3D visualize demo(3DSampleScene.unity).

Graphics.DrawProceduralNow method and the custom shader are used to reduce the overhead time due to data transfer to CPU memory (= ComputeBuffer.GetData method) in the demo script because BlazePoseBarracuda outputs landmarks on the GPU memory (= ComputeBuffer).
Of course, it would be possible to visualize it by generating gameobjects for each landmark after transferring the data to CPU memory.

[creativeIKEP]

I believe this issue is resolved.
I close this issue.