umass-rescue/mlmicroservicetemplate

Concise template to rapidly deploy and share a machine learning model

mlmicroservicetemplate's Issues

Enable GPU For Predictions

Currently, the Docker container used for the microservice only has access to the CPU and memory. Enabling the container to interact with the system GPU can lead to large speed ups in prediction tasks.

Decouple Predictions from Local Network

When creating a prediction request, the images that are passed in are accessed via a local Docker volume shared between the model microservice and the server. While straightforward in its current implementation, this is not the best way to set up the transfer of data since it forces all prediction microservices to be on the same machine as the central server.

Determine how to receive images directly via HTTP request in the /predict endpoint and remove any Docker volume dependencies from the docker-compose.yml file.

Once this is complete, modify the corresponding code in the central server's /model/predict endpoint to send the file via HTTP instead of via filename.

Create Testing UI

Right now when a user wants to test their model, they must either connect via the docker shell or use the server + client to submit requests.

There should be a UI that allows for users to upload images to the model and then view the results.

Predict Batches of Images

Once Issue #7 is complete and images can be received via HTTP, changes can be made to improve the efficiency of predictions.

Images are received and models predict on them one-by-one. However, batching is a more efficient way to handle larger jobs. Determine the best way to predict on batches of images.

(There is a related server issue here)