surya-veer/movement-tracking

UP - DOWN - LEFT - RIGHT movement tracking.

Realtime Face Movement Tracking

90 Lines of code to convert your face movement into keyboard commands.

Description

This is a basic face movement tracking that can convert face movement into keyboard commands like UP - DOWN - LEFT - RIGHT. I used facial landmarks to detect face and get the nose out of it for better referencing. I have created two versions of it, v1 is using a fixed reference boundary which not work expected properly because we need to come at the same position after each movement. To save this I created V2 which uses position change with respect to the previous position. This is more dynamic and easy to control the moves. No need to set position again and again.

movement-v1.py

In version1, I used a fixed reference boundary. If nose reference is out of boundary then I calculate the direction of movement. After getting direction I am converting it into keyboard commands using the keyboard library.

movement-v2.py

In version2, I am using reference change with respect to the previous position in a particular time window and then calculating the direction vector to get direction and converting it to keyboard command.

Dependencies

This is the list of dependencies for running this application. Use pip to install them.

• opencv
• keyboard

$ pip install -r requirements.txt

How to use

1. Download or clone this repository.

2. Extract to some location.

3. First, run movement-v1.py (for fix boundary) or run movement-v2.py(for dynamic movement)
   NOTE: If you are getting 215 assertion failed!! on line 81 check this (#4 (comment))

4. open any online atari game like Subway surfers or temple run.

5. Start doing movements to play game. It will press up-down-left-right based on your movements.

Fun with face movements

Open any online game on the browser which needs UP-DOWN-LEFT-RIGHT movements following games, you can find many games if you search on google.

1. Subway surfer https://www.kiloo.com/subway-surfers/
2. Temple run https://m.plonga.com/adventure/Temple-Run-2-Online-Tablet

You can do a lot more things by the small code change.

** SUPPORT OPEN SOURCE **

movement-tracking's Issues

msmf: init stream failed to set mediaType

I am new to opencv and i found your project of great interest. Please rectify this error that has been raised in the 'assertion failed' code that you uploaded.

Getting a runtime Error

When I run the code it throws this error

Traceback (most recent call last):
File "E:/ml/movement-tracking/movement-v2.py", line 83, in
img, nose_cords = detect_nose(img, faceCascade)
File "E:/ml/movement-tracking/movement-v2.py", line 11, in detect_nose
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

Features

can you say what does detectmultiscale return ?

How is nose detected ?

(-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

Assertion failed in line number 11.

Showing error- error: OpenCV(4.3.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

How do I reduce the window size?

Hello,

Hope you are doing well!

I have a question, how do we reduce the window size? when I run the script, the camera opens and takes the entire screen. How do I reduce that size?

It's not detecting the up and left manner of motion

Having an issue with the code

Traceback (most recent call last):
line 78, in img, nose_cords = detect_nose(img, faceCascade)
line 10, in detect_nose gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-9d_dfo3_\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

Getting a runtime Error

When I run the code it throws this error

Traceback (most recent call last):
File "E:/ml/movement-tracking/movement-v2.py", line 83, in
img, nose_cords = detect_nose(img, faceCascade)
File "E:/ml/movement-tracking/movement-v2.py", line 11, in detect_nose
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

Please help

Need to run as root

I can't run the program without root or with root. and it need to be rooted to aggregate the keyboard(kbd).

.py files closes as soon as it is opened

when i try to run .py file the command prompt window opens for a fraction of second and then closes

low FPS

so there seems to be a significant lag in sensing the video
anyway we can improve the FPS?

Getting error (I am very beginner)

@surya-veer @Yohannfra
please help 🙏

Getting 215 assertion failed!! on line 81!!

Hi , I am newbie to python.I just saw linkedin and try to setup and run this code. getting an error searched Stackoverlow not solved yet.

Traceback (most recent call last):
File "movement-v2.py", line 81, in
img, nose_cords = detect_nose(img, faceCascade)
File "movement-v2.py", line 10, in detect_nose
gray_img = cv2.cvtColor(img, cv2.COLOR_BAYER_BG2RGB)
cv2.error: OpenCV(4.3.0) /Users/travis/build/skvark/opencv-python/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

can you please help?

Thanks in Advance.