dkeras-project/dkeras

Distributed Keras Engine, Make Keras faster with only one line of code.

dKeras: Distributed Keras Engine

Make Keras faster with only one line of code.

dKeras is a distributed Keras engine that is built on top of Ray. By wrapping dKeras around your original Keras model, it allows you to use many distributed deep learning techniques to automatically improve your system's performance.

With an easy-to-use API and a backend framework that can be deployed from the laptop to the data center, dKeras simpilifies what used to be a complex and time-consuming process into only a few adjustments.

Why Use dKeras?

Distributed deep learning can be essential for production systems where you need fast inference but don't want expensive hardware accelerators or when researchers need to train large models made up of distributable parts.

This becomes a challenge for developers because they'll need expertise in not only deep learning but also distributed systems. A production team might also need a machine learning optimization engineer to use neural network optimizers in terms of precision changes, layer fusing, or other techniques.

Distributed inference is a simple way to get better inference FPS. The graph below shows how non-optimized, out-of-box models from default frameworks can be quickly sped up through data parallelism:

Current Capabilities:

• Data Parallelism Inference

Future Capabilities:

• Model Parallelism Inference
• Distributed Training
• Easy Multi-model production-ready building
• Data stream input distributed inference
• PlaidML Support
• Autoscaling
• Automatic optimal hardware configuration
• PBS/Torque support

Installation

The first official release of dKeras will be available soon. For now, install from source.

pip install git+https://github.com/dkeras-project/dkeras

Requirements

• Python 3.6 or higher
• ray
• psutil
• Linux (or OSX, dKeras works on laptops too!)
• numpy

Coming Soon: PlaidML Support

dKeras will soon work alongside PlaidML, a "portable tensor compiler for enabling deep learning on laptops, embedded devices, or other devices where the available computing hardware is not well supported or the available software stack contains unpalatable license restrictions."

Distributed Inference

Example

Original

model = ResNet50()
model.predict(data)

dKeras Version

from dkeras import dKeras

model = dKeras(ResNet50)
model.predict(data)

Full Example

from tensorflow.keras.applications import ResNet50
from dkeras import dKeras
import numpy as np
import ray

ray.init()

data = np.random.uniform(-1, 1, (100, 224, 224, 3))

model = dKeras(ResNet50, init_ray=False, wait_for_workers=True, n_workers=4)
preds = model.predict(data)

Multiple Model Example

import numpy as np
from tensorflow.keras.applications import ResNet50, MobileNet

from dkeras import dKeras
import ray

ray.init()

model1 = dKeras(ResNet50, weights='imagenet', wait_for_workers=True, n_workers=3)
model2 = dKeras(MobileNet, weights='imagenet', wait_for_workers=True, n_workers=3)

test_data = np.random.uniform(-1, 1, (100, 224, 224, 3))

model1.predict(test_data)
model2.predict(test_data)

model1.close()
model2.close()

dkeras's Issues

predict() got an unexpected keyword argument 'callbacks'

File "/root/anaconda3/envs/tf13/lib/python3.6/site-packages/dkeras/workers/worker.py", line 109, in worker_task 
TypeError: predict() got an unexpected keyword argument 'callbacks'

what is tensorflow and keras 's version?
my tensorflow and keras:
keras Version: 2.3.1
tensorflow Version: 1.13.1

Dkeras on a local machine (Intel® Xeon(R) Gold 6128 CPU @ 3.40GHz × 12 ) takes for ever

I have a model based on keras and it takes about 25 seconds for inference. I wanted to speed this up and implemented dkreas but I didnt get the performance improvment that I was expecting. Instead it takes for ever. Do you know why this happen or have I done anything wrong?

from dkeras import dKeras
import ray
ray.init()
model = dKeras(my_model, init_ray=False, wait_for_workers=True, n_workers=None)
segmented = model.predict(data)
model.close()

How to specify different workers if the workers are in the same local networkers but with different IP address

Hello,

I have two workers in Linux or perhaps windows.

They are in the same local network but with different IP address.

If I want to configure distributed keras training, is there any reference for deployment?

Thanks & Regards!
Jun Yan

This is probably a bug

config.N_CPUS_PER_SERVER = n_cpus_per_server
config.N_CPUS_PER_WORKER = n_cpus_per_worker
config.N_CPUS_PER_SERVER = n_gpus_per_worker <------