X-Y coordinates for Mouse Pointer movement

[iitmcvg]

X-Y coordinates for Mouse Pointer movement

[Aravind-Suresh]

Check the oic.cpp file. The lines containing the function calls: draw_eye_gaze and draw_facial_normal are relevant. The draw_eye_gaze function takes the point on the image (pupil) and the gaze vector (Both of them are Kalman-smoothed) along with other parameters. The draw_facial_normal takes the facial-pose vector along with the frame and other parameters.

These are smoothed estimates of gaze and pose respectively. How you use this information to move mouse pointer or any other object for that matter is implementation dependent. You have to control some attribute of the mouse pointer (its position or velocity) using the gaze vector. So the gaze vector (projected onto 2-dimensional space) will be the direction of the instantaneous velocity of the pointer.

We tried something similar to this and it is present in the "demo" branch of this repository. Have a look at it if needed.

Hope this information is useful.

[Aravind-Suresh]

The first argument should be zero. The second argument is the scale factor of mouse movement - try with values between 1 to 100. For this to work you need to uncomment the mouse_move function call.

The demo program also simulates key-presses. Check that also.

[andreosky]

Hi,
thanks a lot for your fast answer. I had already seen the program in the demo branch (and it seems to be just what I'm looking for), but I still have problems to lunch it.
I can successfully compile it (with g++-5 compiler and -fPIC flag). But, when I try to lunch it, I get the following error messages:
"HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP
Beta : 0.855501
Beta : 0.831737
Segmentation Error (core dump created)".
I have no idea on how to solve this problem, but the created core dump file is available at the following link, if it can be useful for you: https://www.dropbox.com/sh/pkgihddzy1ztalr/AACpDsQjHOuXRzqvduT3KLila?dl=0
Thanks a lot again Aravind.
Regards.

[andreosky]

I think that the main problem is that I can't unserstand which parameters I've to give to the main function to launch it!!!

[Aravind-Suresh]

The above bug seems to be some issue related to video drivers. Check if the drivers are proper by running some basic program to read from the camera. If it works do the following, else refer some stack-overflow posts and fix this.

Checkout the v1.0 branch as mentioned in the README. Make the binaries and try executing it. Once the executable works, try implementing the mouse-pointer logic on top of this tagged version. The demo branch might be broken (not sure though) - we were migrating between different versions of OpenCV.

[andreosky]

Now I'm able to run the program. No video drivers problems. Only missing parameters while running it. Please simply help me by explaining the role of the parameters that I need to pass. (For example: I successfully launched it with "./bin/oic 0 1 2" but the mouse pointer doesn't move (even if I enabled the commented line relative to the function mouse_move).

[andreosky]

I already solved that problem, but I want to ask few more questions:
1: Even thought I read that in the function mouse_move there are the eye gaze parameters ("vec_ce_kalman_l" that I think that it means vector_center-eye_kalman_left), it seems that the mouse movement depends only on the nose gaze. Am I right? And, if yes, how to move the pointer with the eye gaze?
2: It seems that mouse moves with relative coordinates. How to move the pointer with absolute position? Which values should I pass to XWarpPointer?
3: How to simulate a click of the mouse with my eyes? Is it already implemented?
I'm glad to talk with a brilliant person like you, you are very kind.

[andreosky]

Aravind-Suresh please help me by answering to the last message..then I'll close the issue..
Thank you again for your patience