mahmudulalam/unified-gesture-and-fingertip-detection

Unified learning approach for egocentric hand gesture recognition and fingertip detection.

I am Mohammad Mahmudul Alam, Ph.D. candidate in Computer Science at UMBC. My ongoing research involves neuro-symbolic representations and their applications to deep learning. Currently, I am working as a Graduate Research Assistant in the CORAL lab at UMBC where I am advised by Professor Dr. Tim Oates and co-advised by Dr. Edward Raff.

Linked In 🔗       Google Scholar 📚       Contact Me! 📬

 

unified-gesture-and-fingertip-detection's Issues

Custom Model keyword arguments Error

Change
model = Model(input=model.input, outputs=[probability, position])
to
model = Model(inputs=model.input, outputs=[probability, position])
on line 22 of net/network.py

OSError: Unable to open file (unable to open file: name = 'yolo.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

I use the Mac Os to run thereal-time.py file, and get the OSError, I also search on Google to find others' the same problem. It is probably the Keras problem. But I do not how to solve it

Using old versions of tensorflow, can't install the dependencies on my macbook and with newer versions it's constatly failing.

When trying to install the required version of tensorflow:

pip3 install tensorflow==1.15.0
ERROR: Could not find a version that satisfies the requirement tensorflow==1.15.0 (from versions: 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.2.1, 2.2.2, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0, 2.3.1, 2.3.2, 2.4.0rc0, 2.4.0rc1, 2.4.0rc2, 2.4.0rc3, 2.4.0rc4, 2.4.0, 2.4.1)
ERROR: No matching distribution found for tensorflow==1.15.0

I even tried downloading the .whl file from the pypi and try manually installing it, but that didn't work too:

pip3 install ~/Downloads/tensorflow-1.15.0-cp37-cp37m-macosx_10_11_x86_64.whl
ERROR: tensorflow-1.15.0-cp37-cp37m-macosx_10_11_x86_64.whl is not a supported wheel on this platform.

Tried with both python3.6 and python3.8

So it would be great to update the dependencies :)

Difference of classes5.h5 and classes8.h5

Hi, May i know the difference when training classes5 and classes8? are the difference from the dataset used for training by excluding SingleSix, SingleSeven, SingleEight or there are other modification such as changing the model structure or parameters?

Thanks

About datasets

how to get the files like train_y_keys.npy and train_y_prob.npy?I may be careless and i cant find these in code.
thank you for answering

How to change model to only classify 2 finger and 2 classification of gesture

I need to change the model so that it only detects 2 classes and 2 fingers, which output layer do i need to change so i can do this?

Problem of weights

Hi,when load the solo.h5(In solo.py line 14:"self.model.load_weights(weights)") it will report errors:
Process finished with exit code -1073741819 (0xC0000005)
keras2.2.5+tensorflow1.14.0+cuda10.0

OSError: Unable to open file (unable to open file: name = 'weights/performance.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = 'weights/performance.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

Datasets

Hello，
I have a question about the dataset from your readme, I can't download the Scut-Ego-Gesture Dataset ，Because in China, this website has been banned. Can you share it with me in other ways? For example, Google or QQ email: [email protected]

can use mobilenet and run on mobile devices?

how to process the points that move out when do scale translate or rotation augment op?

when use imgaug to generate image batch, why no logical to process points that move out image condition?

tensorflow-gpu package is no longer supported

Proposed Solution

1. Update the project's dependencies by replacing "tensorflow-gpu" with "tensorflow" in the project's requirements or setup files.
2. Test the project to ensure it functions correctly with the new package.
3. Update the project's documentation to reflect this change.

why are there two hand detection provided?

A wonderful work!!As mentioned above, the Yolo and Solo detection models are provided. I wonder what is the advatange of each model comparing to the other and what is the dataset to train the detect.

left hand?

Hi, first it's really cool work!

Is the left hand included in the training images? I have been playing around with some of my own images and it seems that it doesn't really recognize the left hand in a palm-down position...

If I want to include the left hand, do you think it would be possible if I train the network with the image flipped?

cannot download the pretrained weights

The download process always stops in the mid way. Is there any other download sources? Thank you very much.

What python version does this use?

how to download the weights, code not contain?

The weights folder contains three weights files. The comparison.h5 is for first five classes and performance.h5 is for first eight classes. solo.h5 is for hand detection. but no link