juba-agoun/fleet

Online Federated Learning.

FLeet

Author: Georgios Damaskinos ([email protected])

FLeet is a middleware introduced in FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction.

Quick Start

The following steps train a CNN on the non-IID MNIST dataset.

1. Setup
   • Ubuntu 18.04.3 LTS
   • gcc 7.5
   • Python 3.6
   • Java 1.8.0_77
   • Maven 3.6.3
   • Gradle 4.10.1
   • Android API level 25
   • Create and start an Android emulator
   • Export variables:
     • JAVA_HOME
     • ANDROID_HOME
   • Download Python numpy package
2. Download and extract the 4 files of the MNIST dataset
3. Compile, deploy and run FLeet

   ./local_deploy.sh 5554 9992 ~/log /path/to/mnist/ 0 0 1000 3000 0.0005 1 1 0 0 0 0 0
   

Detailed information about the deployment options for FLeet is available here.

Usage

The following contains information about each module of FLeet along with instructions for implementing a new application.

Server

Performs the descent computation and updates the model

• Edit SPUpdater.java and SPSampler.java
• Set profiling method in MasterOrchestrator.java

Driver

Initializes the Server and performs a periodic evaluation

• Edit Driver.java

Android Client

Performs the gradient computation

• Edit SPGradientGenerator.java

Existing applications

cppNN

Image classification application based on Convolutional Neural Networks and written in C++.

• Enable/disable compression
  • Enable compressed SGD: #define DISTILLATION_MODE 1
  • Disable compressed SGD: #define DISTILLATION_MODE 0
  • Files: Server cppNN_backend.cpp, Driver cppNN_backend.cpp, Client cppNN-lib.cpp, network.h

MLP

Image classification application based on multilayer perceptron and written in Java.

LR

Facebook check-in prediction application based on logistic regression and written in Java.

DL4J

Port of DL4J library on FLeet.

Datasets

MNIST

See Quick Start.

EMNIST

• Download Python dependencies:
  • Sklearn
  • Scipy
  • Numpy
• python emnistParser.py --help
• For parsing into FLeet change to the DL4J app:
  • See all TODO notes in the Driver.
  • Change the application in the Server and Client (see Usage).
  • Give the output of the parser as input dataset for FLeet.

CIFAR

• Download CIFAR-10 from here or CIFAR-100 from here and extract.
• For CIFAR-100:

  cd cifar-100-binary/
  mv train.bin data_batch_1.bin
  mv test.bin test_batch.bin

• For parsing into FLeet:
  • Comment out these lines and comment in these lines. Choose this line for CIFAR-100 or this line for CIFAR-10.
  • Comment out these lines and comment in these lines for CIFAR-100 or these lines for CIFAR-10.
  • Comment in this line for both datasets.