yuanyuanli85/keras-multiple-process-prediction

A Simply Example to show how to use Keras model in multiple processes to do the prediction

I am a Machine Learning Engineer, woking on AI Asic. Focus on neural network pruning and low-bit quantization to acclerate inference fps and latency.

Research

• Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks

Github Stats

keras-multiple-process-prediction's Issues

Model Inference on Video

What is needed to be change if I want to do model inference on a video file instead of images? I am using cv2.videoCapture(video_path) for reading the video, then I throw a while loop when the cap is opened. Do I still need queue the image from cap.read() function or anything else to be done?? Also, I know that I have to change start function, but I dont know exactly what I should do. Thanks in advance.

Training

How would you write a function to do Training instead of predictions?

Load model multiple times

Hi guys.
I really appreciate your project. It helps me a lot.
But I have a problem here. In my case, my img_path folder changes all the time.
Take example, at first I load all the images in that folder to predict using your code.
During predicting time, more and more images update to that folder and I have to predict again after finish the previous step.
So the problem is that I have to load model again after finish 1 step. Is there any way to load model only one times ?
Thank you in advance

How to adopt this code in multiprocessing CPU instead of GPU

Hi,
Thank you for this wonderful example. Please guide me how I can adopt this code in multiple CPU for prediction.

Thanks,

Get result for all the files

Hello,

Thanks for the project, I tried to implement in my project and it works but one issue I am facing and its very silly.

how do I get the output for all the images present in imagepath from vgg16_worker to main program after worker.join? I tried a couple of options I mean creating a result_queue and put the outputs there but the queue is having result of the only one image.

CUDA_ERROR_NOT_INITIALIZED

I follow the code and implement my own. When I call keras.models.load_model() in the child process, I got the following errors:

E tensorflow/stream_executor/cuda/cuda_driver.cc:1287] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED
F tensorflow/stream_executor/cuda/cuda_driver.cc:312] Check failed: CUDA_SUCCESS == cuCtxSetCurrent(cuda_context->context()) (0 vs. 3)