karthikeshwar1/gra

Gesture Recognition App

GRA

A program which performs specific actions based on recognized gestures.

This repository is a submission to the REVA HACK</>2020 by team "awake ? code : sleep"

✨ Features

• Detect gestures: To detect hand gestures with acceptable accuracy
• Carryout function: Perform keystrokes when a gesture is detected

Gestures:

one

swipe_up

swipe_down

pinch_in

pinch_out

two_fingers_left

two_fingers_right

🕹️ Usage

pip install -r requirements.txt

python gra.py

Gestures with their functions:

One : toggle gesture detection

Swipe up -> swipe down : show task view -> show applications

Swipe down -> swipe up : show desktop -> show applications

Pinch in/out: Zoom in/out

Two fingers swipe left/right : Move to virtual desktop left/right

🔩 Behind the scenes

• Our application identifies the gestures through images through image classification
• This image classifier is obtained from training the MobileNetv2 model
• Around 1000 images have been trained for each of the 7 image classes
• When it detects a specific gesture from the webcam, it performs action assigned to that gesture
• The Midas touch problem (inability to turn off gestures) is solved by providing a gesture (one) to toggle the application
• The program also doesn't take too many inputs in a row

🌱 Applications

• This can help us use the computer in a much more accessible way
• It is also very helpful to differently abled people
• Future is going to be dominated by mixed reality, smart glasses, etc. Gestures are the main way to interact with such technologies
• Playing games becomes more fun

🔨 Tools Used

• Python
• Tensorflow for Image-classification using ML.
• OpenCV for image manipulation.
• Plyer for notifications.

📄 License

👨‍💻 Contributors

Karthikeshwar

Guruprasad V

Siddhanta Mandal

Vikas G N